$$\frac{\mathbf{a}}{\mathbf{c}}$$

**The Science-Media Interface**

## **Knowledge and Information**

Studies in Information Science

Edited by Wolfgang G. Stock

Editorial Board Ronald E. Day (Bloomington, Indiana, U.S.A.) Richard J. Hartley (Manchester, U.K.) Peter Ingwersen (Copenhagen, Denmark) Michel J. Menou (Les Rosiers sur Loire, France, and London, U.K.) Stefano Mizzaro (Udine, Italy) Christian Schlögl (Graz, Austria) Sirje Virkus (Tallinn, Estonia)

# **The Science-Media Interface**

On the Relation Between Internal and External Science Communication

Edited by Irene Broer, Steffen Lemke, Athanasios Mazarakis, Isabella Peters, and Christian Zinke-Wehlmann

Diese Publikation wurde unterstützt durch den Open-Access-Publikationsfonds der Universität Leipzig und den Publikationsfonds für Monografien der Leibniz-Gemeinschaft. | This publication was supported by the Open Access Publication Fund of Leipzig University and the Leibniz Open Access Monograph Publishing Fund.

ISBN 978-3-11-077636-2 e-ISBN (PDF) 978-3-11-077654-6 e-ISBN (EPUB) 978-3-11-077663-8 ISSN 1868-842X DOI https://doi.org/10.1515/9783110776546

This work is licensed under the Creative Commons Attribution 4.0 International License. For details go to https://creativecommons.org/licenses/by/4.0/.

Creative Commons license terms for re-use do not apply to any content (such as graphs, figures, photos, excerpts, etc.) not original to the Open Access publication and further permission may be required from the rights holder. The obligation to research and clear permission lies solely with the party re-using the material.

#### **Library of Congress Control Number: 2023944182**

#### **Bibliographic information published by the Deutsche Nationalbibliothek**

The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data are available on the Internet at http://dnb.dnb.de.

© 2023 with the author(s), editing © 2023 Irene Broer, Steffen Lemke, Athanasios Mazarakis, Isabella Peters, and Christian Zinke-Wehlmann, published by Walter de Gruyter GmbH, Berlin/Boston. This book is published with open access at www.degruyter.com.

Printing and binding: CPI books GmbH, Leck

www.degruyter.com

### **Contents**

Irene Broer, Steffen Lemke, Athanasios Mazarakis, Isabella Peters, and Christian Zinke-Wehlmann  **Editorial: The Science-Media Interface – On the relation between internal and external science communicationVII**

Enrique Orduña-Malea and Rodrigo Costas  **1. A scientometric-inspired framework to analyze EurekAlert! press releases1**

Irene Broer **2. Curating, transforming, constructing science news: The newsmaking routines of Science Media Center Germany29**

Arno Simons and Alexander Schniedermann  **3. Preprints in the German news media before and during the COVID-19 pandemic. A comparative mixed-method analysis53**

Max Brede, Athanasios Mazarakis, and Isabella Peters  **4. What drives researchers to look up research publications they found in the news?79**

Steffen Lemke, Athanasios Mazarakis, and Isabella Peters  **5. Path model of the interplay between the promotion and the received attention of research articles117**

Markus Lehmkuhl, Nikolai Promies, and Melanie Leidecker-Sandmann  **6. Repercussions of media coverage on science? A critical assessment of a popular thesis139** 

**Name Index161**

**Subject Index169** 

### Irene Broer, Steffen Lemke, Athanasios Mazarakis, Isabella Peters, and Christian Zinke-Wehlmann **Editorial: The Science-Media Interface – On the relation between internal and external science communication**

The publication and distribution of scientific results is of major importance for knowledge societies (Stehr, 1994; Castelfranchi, 2007), especially in the face of the complex and multifaceted challenges in today's world. This volume takes as its starting point a twofold interest in the communicative interplay between science and the mass media. First, the ways in which "scientific facts" as the result of scientific research, discourse, and shared conventions (Fleck, 2019 [1935]) become part of public communication, especially through journalism. Second, the ways in which public communication about, and especially journalistic representations of and references to, scientific knowledge affect processes of knowledge production, scientific discourse, and allocation of reputation within science. Major actors in this interface are researchers themselves, professional science communicators and science journalists, but also platforms and intermediary organizations that curate scientific research for distribution into mass media. Each of these have their own approach to the selection, presentation and mediation of scientific knowledge. To highlight different aspects of the science-media interface, this volume integrates perspectives from scientometrics and quantitative science studies, and from communication science and journalism studies.

The concepts of *internal* and *external science communication* are useful for distinguishing the communication practices by which scientific knowledge is produced, verified, shared, and acknowledged within the scientific community from those by which it is communicated and engaged with outside of the scientific community (Leßmöllmann & Gloning, 2019). The rise of digital media has, however, led to a refiguration of science communication, characterized by new actors, practices and orientations (Broer & Hasebrink, 2022) and a blurring of traditional role divisions between the production, evaluation and dissemination of science (Franzen, 2019; Neuberger et al., 2019). As the works in this volume highlight, the boundaries between internal and external science communication are permeable, resulting in deeply interwoven relationships affecting both forms of science communication.

Within the domain of science, the scholarly community uses distinct practices of internal science communication to produce, distribute and verify scientific knowledge in what is typically referred to as "scholarly communication" (Borgman, 1989, p. 586). Fields such as quantitative science studies or scientometrics invest great efforts into understanding how scientific knowledge is produced and exchanged within and between scientific communities (and in some cases between academia and industry or policy, see for instance Leydesdorff & Meyer, 2003). This is often driven by the objective of quantifying impacts that indicate the influence or relevance of academic journals or individual research endeavors.

So far, the field of quantitative science studies has remained largely introspective. Increasing interest in the consequences of open science, and potential indicators for measuring the societal impact of research (Tahamtan & Bornmann, 2020) may reflect a shift. However, even within the extensive body of literature on research impact indicators (e.g., citations, usage metrics, altmetrics), research into the scientific impact of external science communication, e.g., in terms of science reporting, blogging or social media activities, seems to still be in its infancy. Although the exact nature of the relationship has remained unclear, limited studies do indicate a connection between mentions of scientific publications in news articles and social media platforms, and their scientific impact (see, e.g., Fanelli, 2013; Dumas-Mallet et al., 2020; Phillips et al., 1991). So, given the variety of formats from external science communication only more influencing factors and relationships seem to be awaiting exploration.

The transformations that occur when representations of scientific knowledge move beyond the boundaries of science into other social contexts have been described as "popularization" (Fleck, 2019 [1935]; Hilgartner, 1990). It can be argued that this process occurs on a continuum: on the way from the esoteric knowledge communities where scientific knowledge was originally generated, it is communicated with increasing certainty in order to make abstract ideas concrete (Bauer, 2017; Cloitre & Shinn, 1985; Hilgartner, 1990). The news media have traditionally played an important role in the public communication of science, which is why research on external science communication has often focused on the role of journalism. In some normative models of science communication, the news media are idealized as conduits and problematized as distorters in the transmission of scientific knowledge to the wider public (Mede & Schäfer, 2020). In this view, journalistic practices associated with making scientific knowledge fit for public consumption are linked to concerns about simplification, decontextualization and a loss of accuracy (Berg, 2018). However, as journalism-oriented research has pointed out, news media operate according to their own logics

(Kohring, 2005). The practices within science reporting and the quality of journalistic representations of science should therefore be analyzed according to the professional norms of journalism, not science. In this view, science journalists take on a range of societal roles from gatekeeping, contextualizing, to critically observing science (Fahy & Nisbet, 2011).

The interface between science and media is, however, affected by several trends. On the side of science, researchers are facing pressure to increase their scientific output in order to positively impact the quantitative metrics that are used to determine academic standing and which are necessary for career progression (i.e., "publish or perish"). In addition, there is a usually normatively argued push towards open access in academic publishing (Taubert et al., 2019) and towards sharing of research results as early as possible. While not new, sharing scientific findings before completing the academic peer review process in socalled preprints has gained traction, particularly with regard to COVID-19 research (Fraser et al., 2021). In sum, there is not only a surge in scientific output, but also in outlets where scientific findings can be freely accessed.

Individual researchers and research institutions are furthermore increasingly engaging in external science communication practices, e.g., via science blogging, social media activities, as well as press announcements about new research. On the one hand, these efforts are a way to fulfill normative expectations, such as sharing scientific knowledge to inform citizens and politicians, and justify public expenditures on scientific research (Renn, 2017). On the other hand, strategic science communication helps scientific actors gain public visibility, which in turn may enhance the reputation of research institutions and individual scientists, and attract funding (Weingart & Joubert, 2019; Väliveroonen, 2021; Raupp, 2017).

At the same time, traditional mass media are undergoing rapid change due to digital communication technologies. A loss of advertising and subscription revenue has led to budget cuts and a greater reliance on non-specialist and freelance journalists (Dunwoody, 2021). In the face of the abovementioned surge in scientific output and outlets, and scientific actors' increased efforts to gain public visibility, shorter production times and a lack of specialists in journalism have brought about concerns about the quality of science reporting, and the extent to which journalistic roles are being fulfilled. Studies into so-called "churnalism" have found significant reliance on press releases from universities and journals in science reporting (Heyl et al., 2020; Vogler & Schäfer, 2020).

The pursuit of media attention by scientific actors can also have implications for science itself, as the discourse on the mediatization of science suggests (see

Rödder et al. 2012 for an overview). This research area is concerned with the effects of tendencies within science to orient scientific research and publication processes towards journalistic criteria of relevance in order to gain media attention, rather than towards internal scientific criteria of relevance. Finally, it is interesting to note that scientists and journalists appear to be facing many of the same constraints: a push to produce more content in less time and the need to search, select, verify, contextualize and evaluate a rapidly growing amount of scientific output that is available in ever more outlets.

#### **Content summary**

The order in which the individual chapters within this volume are presented mirrors an exemplary chronological sequence in which scientific and journalistic publishing may affect each other. In this sequence, a bulk of findings from the scientific sphere enters the journalistic sphere via press releases from academic publishers or institutions (Chapter 1). In navigating this information, intermediaries like Science Media Centers provide journalists with curated science news and expertise (Chapter 2), while preprints give journalists early access to new, though unverified, scientific findings (Chapter 3). The resulting media coverage may in turn affect the degree of attention given to research within the field of science (Chapter 4): A feedback mechanism with potential implications for the methods that scientists rely on to evaluate research (Chapter 5). The final chapter concludes this volume with a critical discussion of such feedback mechanisms' plausibility along the concept of medialization (Chapter 6).

All chapters within this volume were peer-reviewed individually. Several of the chapters originated from the research project "MeWiKo - Medien und wissenschaftliche Kommunikation [Media and Scholarly Communication]", which was funded over three years by the German Federal Ministry of Education and Research – this refers to the Chapters 2, 4, and 5. This edited volume can therefore also be considered a representation of some of the MeWiKo-project's core results, which in this publication get embedded and contextualized within recent works of other contributors stemming from scientometrics, science studies, communication science, and journalism research.

In this volume's first chapter, *Orduña-Malea & Costas (Chapter 1)* contribute to opening up one of the most common formats of (external) science communication – press releases – for the field of scientometrics. The authors present a theoretical framework for the analysis of science-related press releases, based on principles common in scientometric research. As an illustrative example, the authors perform a descriptive analysis of the press release data available on EurekAlert.org, its coverage regarding press release types, disciplines, years, continents, submitters, and journals, as well as of the web- and Twitter-links to EurekAlert! press releases in the past. Within their chapter, Orduna-Malea & Costas present a helpful theoretical assessment of EurekAlert! in the context of science communication. At the same time, their empirical results provide interesting insights into the data behind the indubitably important player that EurekAlert! has become in the dissemination of science news.

In the second chapter, *Broer (Chapter 2)* investigates how Science Media Centers as important but understudied intermediaries between science and journalism, contribute to the construction of science news. By reporting results from an ethnographic study of Science Media Center Germany, Broer outlines the routines with which this organization curates scientific knowledge and expertise for journalistic distribution. These include restricting the scope of covered scientific topics, applying scientific, social, and journalistic relevance criteria, relying on external expertise, and timing broadcasts with the intention of promoting, altering, or preventing coverage of science issues. She argues that in this process, the editors negotiate implicit and explicit knowledge about science and journalism in an effort to bridge the professional norms of these fields.

*Simons and Schniedermann (Chapter 3)* continue with a further chapter strongly related to the COVID-19 pandemic's effects on science journalism. The authors present a systematic case study on the coverage and framing of scientific preprints within a large sample of German news stories over the years directly before and during the pandemic. Simons & Schniedermann display a rich collection of both quantitative and qualitative analyses, which, among other aspects, illustrate the heavy reliance of journalism on preprints in COVID-19-related news stories since the beginning of the pandemic. Their findings also provide highly interesting insights into how German journalism tended to frame preprint-based information with regard to dimensions such as uncertainty or timeliness, and how such framings appeared to change over time.

In this volume's next part, *Brede, Mazarakis and Peters (Chapter 4)* approach the topic of how journalistic stories on research might affect the attention that covered research articles themselves receive. More precisely, the authors utilize an approach based on conjoint analysis to examine which features of scientific articles, if portrayed in journalistic coverage, may lead to researchers looking up said original articles after encountering them within public media. Brede et al. hereby investigate potential drivers that could explain associations between science's presence in journalism and metrics commonly used to evaluate scientific

impact. The results indicate that the features with an inferred external judgment were the most useful. In addition, one of the most important elements influencing the effect advantage may be the supplementary, thematically, and methodologically categorized information that a news article may provide.

The subsequent chapter by *Lemke, Mazarakis and Peters (Chapter 5)* expands upon this connection between research's coverage in external science communication and impact metrics. Moreover, the authors provide a concrete example for an empirical analysis examining press releases in a scientometric context (see also Chapter 1). Lemke et al. combine press release data from EurekAlert! with data on embargo e-mails, bibliometric indicators, and altmetrics, to estimate path models that indicate the substantial interdependencies between the presence of research articles in science communication and metrics commonly used as indicators of impact. The large-scale approach by Lemke et al. reveals the significant degree to which science communication and scientometric impact metrics are linked to each other.

In this volume's last chapter, *Lehmkuhl, Promies and Leidecker-Sandmann (Chapter 6)* assess a much-discussed claim about the dynamics between science and journalism: the thesis of the medialization of science, or more specifically, the included assumption that journalism in a kind of feedback mechanism affects the practices of scientific actors, which, according to the medialization thesis, in turn increasingly adapt to journalistic selection logics. Lehmkuhl et al. compile and discuss various studies' findings concerning one central premise of this assumption, namely journalism's performance in creating and concentrating public attention for science. From their evaluation, the authors conclude that the empirical studies provide hardly any evidence for journalism's success in focusing public attention on certain scientific results or actors – which, as Lehmkuhl et al. argue, would be a prerequisite of central importance for the plausibility of the medialization thesis. With their findings, the authors provide thought-provoking new input to the discussion of one of the over the past decades arguably most prevalent theoretical conceptions of the relationship between science and journalistic media.

#### **Contribution**

The chapters presented within this volume cover a wide range of approaches, from theoretical considerations on the interplay of academic and journalistic communication of science to practical applications of related data sources. Likewise, the main learnings from the presented studies touch upon a large variety of aspects related to the science-media interface, e.g., how scientometric methods can be utilized to gather insights from and about outputs of external science communication, what the processing of scientific content within journalistic media currently looks like and how it changed during the COVID-19 pandemic, and how researchers might react to the way external science communication depicts science. Thus, it is one of the first volumes that has successfully collected and contextualized up-to-date research from a variety of disciplines that deal with the journalistic and scientific spheres and the sphere in which science and media meet and overlap. The volume sets a particular spotlight on the changing practices of internal and external science communication induced by different forms of publishing and communicating scientific results, such as preprints, press releases, and embargo e-mails, and how those affect (science) journalism and quantitative science studies. We thus believe this edited collection to offer content relevant for a variety of readers, including but not limited to researchers and students from the fields of science communication, scholarly communication, the science of science, science journalism, bibliometrics, information science, journalism studies, communication science, or sociology of science; as well as other stakeholders from, for instance, domains of science policy.

This volume's contents also indicate the abundance of opportunities for further research at the intersection between subject fields traditionally dealing with the analysis of scientific systems of knowledge creation (e.g., scientometrics, sociology of science, science and technology studies) and those researching the workings of journalism and public communication of science (e.g., media and communication studies, journalism studies). Throughout this volume, a recurring observation in this regard concerns the remaining need for more research on the factors and mechanisms that guide actions and decisions by the diverse stakeholders involved in the public communication of research findings. How, for instance, do press offices select research publications for their promotional activities, to which degree are traditional news values applicable to science journalism (see also Franzen, 2014; Badenschier & Wormer, 2012), how do journalists reflect their increasing use of preprints (see also Fleerackers et al., 2022), how do researchers perceive accounts of science within external science communication, etc.. In many instances the contributions within this volume indicate (and demonstrate) how the aforementioned fields of research can benefit from more extensive mutual exchange, either on the level of theories, models, methodologies, or data sources. Moreover, the amount of remaining research demand revealed within the chapters underlines that such exchange will remain beneficial in the long run.

Finally, it should be noted that the majority of studies presented in this volume were performed during the COVID-19 pandemic – a time when the relationship between science and media received increased public attention as it repeatedly became the subject of discussion within public media. Starting from the early days of the pandemic, the normative roles of journalism in disseminating scientific findings to non-academic audiences were discussed as openly as perhaps never before. Researchers, on the other hand, in many examples demonstrated the immediacy with which their insights can nowadays be brought into the public discourse, be it via postings or open letters shared across social media, or via rising scientific publication formats such as podcasts or preprints (Watson, 2022; see also Chapter 3). Among many other things, the COVID-19 crisis has highlighted the importance for both journalists and researchers to exhibit responsibility in their public communication of research, and thus the necessity for the scientific and the journalistic sphere to profoundly understand each other's modes of communication. We hope that this volume dedicated to previously under-researched phenomena unfolding at the science-media interface will contribute to such understanding.

As the editors of this volume, we wish to thank the Leibniz Open Access Monograph Publishing Fund and the Open Access Publication Fund of Leipzig University for funding this open access publication. We also thank the chapters' authors for their contributions and close collaboration during this volume's creation, Marie Wilke for her support in the editing process, and De Gruyter Saur for their fruitful cooperation.

> Irene Broer, Steffen Lemke, Athanasios Mazarakis, Isabella Peters, and Christian Zinke-Wehlmann July 2023

### **References**





### Enrique Orduña-Malea and Rodrigo Costas **1. A scientometric-inspired framework to analyze EurekAlert! press releases**

**Abstract:** Press releases about scholarly news are brief statements provided in advance to the press, including a description of the most relevant findings of one or more accepted scientific publications, usually under the condition that journalists will adhere to an embargo until the publication date. The existence of centralized platforms such as EurekAlert! allows press releases to be disseminated online as independent news articles. Press releases can include additional material (e.g., interviews, commentaries, explanatory tables, figures, media, recommended readings), which turn them into online objects with analytical value of their own. The objective of this work is to illustrate how press releases can be quantitatively analyzed applying similar tools and approaches as those applied in scientometric research (SCI). To achieve this goal, a scientometricinspired analytical framework is proposed based on the formulation of spaces of interaction of objects, actors, and impacts. As such, the framework proposed considers press releases as science communication (SCO) objects, produced by different SCO actors (e.g., journalists), and the subject of receiving impact (e.g., tweets, links). To carry out this analysis, all press releases published by EurekAlert! from 1996 until 2021 (455,703 press releases), all tweets including at least one URL referring to a EurekAlert! press release (1,364,563 tweets), and all webpages with at least one URL referring to a EurekAlert! press release (54,089,233 webpages) have been studied. We argue that the large volume of press releases published and their online dissemination make these objects relevant in the measurement of SCO-SCI interactions.

**Keywords:** scientific news, press releases, science communication, altmetrics, webometrics, EurekAlert!, scientometrics

### **Introduction**

The communication of scientific results is an intrinsic part of the scientific endeavor. Scientists typically follow a formal communication channel regulated by a peer-review process. The basic unit of scholarly communication (communication of science within science) is the writing of a scientific publication which is published and publicly disseminated after peer review (Latour & Woolgar, 1986 [1979]). Other communication channels (Garvey, 1979) are also used to disseminate scientific research results in a more flexible way, and often also to broader audiences (e.g., preprints, scientific events, science fairs, distribution lists, media appearances, blogposts, museum exhibitions, etc.). The audience (scientists, mediators or the general public) can re-disseminate these research results under different forms to accomplish a number of goals, thus expanding the scope of what is commonly referred to as science communication (communication of science outside science).

Science communication is defined as the use of appropriate skills, media, activities, and dialog to enhance public awareness (e.g., creating familiarity with new aspects of science), enjoyment or other affective responses (e.g., appreciating science as entertainment or art), interest (e.g., voluntary involvement with science or its communication), opinion-forming (e.g., the forming, reforming, or confirming of science-related attitudes), and understanding (e.g., content, processes, and social factors) of science (Burns et al., 2003).

The increasing diversity of voices communicating science (Vogler & Schäfer, 2020) makes the role of science communication mediators (especially journalists) of special importance. News media routinely report on findings and discoveries newly published in scientific journals via science news (Kiernan, 2003; Groves et al., 2016).

News media facilitate the transmission of discoveries to citizens, enhancing thereby the public understanding of science (Autzen, 2014; Stockton, 2016) and favoring accountability of the public investment in science. Journalists can also act as watchdogs towards the influence of government and industry on scientists (de Vrieze, 2018). However, mass media are addressed to large unspecific audiences and a medialization of science effect can distort the scientific message (Weingart, 1998, 2012; Franzen, 2012; Franzen et al., 2012). The establishment of editorial decisions primarily based on news values and trendiness instead of the intrinsic publications' scientific value might favor the writing of new articles oversimplifying research results, exaggerating findings, hiding problematic information, or monitoring just a few highly selective multidisciplinary journals (Woloshin & Schwartz, 2002; Fanelli, 2012).

Media exposure could also alter the citation-based impact of publications (see also Chapters 4 and 5 within this volume). In this regard, the scientific literature has addressed two main hypotheses, the "publicity hypothesis," which assumes that media coverage genuinely increases the academic citations of the publications portrayed, and the "earmark hypothesis," which assumes that media coverage merely earmarks outstanding articles which would have received many citations anyway (Phillips et al., 1991). Earlier literature has proved

that publications disseminated in the media obtained on average higher numbers of citations than those publications not covered by the media (Phillips et al., 1991; Kiernan, 2003; Fanelli, 2012; Alonso-Flores et al., 2020; Dumas-Mallet et al., 2020). Lemke et al. (2022) have also shown that articles mentioned in embargo e-mails receive higher citations. However, no causality has been established.

The scientific community in general – and science journal publishers in particular – have been traditionally reluctant to the informal dissemination of research results in the media, especially before peer review. The well-known Ingelfinger rule (the policy of considering a manuscript for publication only if it has not been submitted elsewhere, particularly through the popular press) constitutes a milestone in the control of science communication flows (Kiernan, 2006). Journal embargoes constitute another form of control on the dissemination of science news (Kiernan, 1997). To prevent the pre-publication of results in the media, journal editors alert science journalists about new original articles they deem remarkable a few days prior to the publication date (Franzen et al., 2012; Franzen, 2012). To do this, journals provide advance copies of the publications and distribute press releases (brief statements given to the press including a description of the most relevant findings of one or more accepted publications) under the condition that the journalists will adhere to a strict embargo until the publication date. This way, journal editors warrant that any new research has been properly peer-reviewed before being presented to the lay public, while they provide journalists enough time to write science news accurately (Stockton, 2016).

Web technologies allowed going one step further in the development of the embargo system, and online centralized platforms such as AlphaGalileo1 or EurekAlert!2 were launched. These services allow not only journals but also other research bodies (mainly universities) to submit detailed press releases which will be delivered to the journalists subscribed to the service. These press releases are usually elaborated by trained journalists working in professional press offices and can include personal interviews with the authors and independent third-party commentaries, supplementary materials, recommended readings, and other informative elements. When the embargo for scientific publications has finished, and regardless of the re-dissemination of the embargoed information through media, the press releases are directly published online by the news service website and disseminated via social networking sites.

 **1** https://www.alphagalileo.org/en-gb

**<sup>2</sup>** https://www.eurekalert.org

### **EurekAlert! as a press releases data source**

EurekAlert! is a non-profit service established in May 1996, initially homed in the Stanford University servers and later moved and operated by the American Association for the Advancement of Science (AAAS) as a centralized online hub for science press releases (Stockton, 2016).

EurekAlert! only accepts contributions (press releases) from Public Information Officers (PIOs) at organizations that conduct, publish or fund scientific research in all scientific disciplines (there is no limit to the number of PIOs from one organization). These organizations must meet several eligibility criteria3 and pay an annual fee. Journalists, who also need to meet specific eligibility criteria4 , might apply for free access to embargoed press releases submitted by PIOs. Only legitimate content owners may designate an embargo date/time for journalists when submitting a press release. EurekAlert! accepts specific press releases categories following specific restrictions. The scope and coverage of each press release category is provided in the supplementary material (Appendix A).

EurekAlert! includes specific channels including press releases written in French, German, Spanish, Portuguese, Japanese, and Chinese (which are also published in English), and holds specific news channels from science agencies (US Department of Energy, US National Institutes of Health, and US National Science Foundation) as well as general interest topics (climate change, cancer research or marine science). EurekAlert! has been questioned due to indirect effects both on journalists and PIOs (de Vrieze, 2018). First, some media have ended up replicating press releases without added value (Schäfer, 2017). Second, universities have professionalized their press offices and intensified their communication towards news media (Vogler & Schäfer, 2020), limiting the direct contact of scientists with journalists, and using press releases as effective means of communicating science and controlled tools to show utility (Autzen, 2014), overstating the societal implications of their findings (Franzen, 2012) to get attention and build reputation. These issues have raised concerns about the EurekAlert! model, which has come to be referred to as a marketplace (de Vrieze, 2018).

Scientometrics have paid little attention to EurekAlert! as an object of study. Bowman and Hassan (2019) developed the only descriptive analysis to date,

 **3** https://www.eurekalert.org/pioguidelines

**<sup>4</sup>** https://www.eurekalert.org/reporterguidelines

describing the coverage of EurekAlert! in the Altmetric.com database. Lemke et al. (2021) analyzed qualitative aspects of EurekAlert! press releases (structure, linguistic accessibility, and the existence of narratives) and their potential influence on the impact of the publications promoted.

While the authors acknowledge that science communication is not a single concept or construct, but a communication activity that involves multiple aspects and components that take place in multiple channels/modes from numerous sources, this study aims to examine press releases as specific quantifiable objects with the potential to enable the measurement of science communication interactions, an issue that has not been addressed so far.

### **A scientometric-inspired framework to study press releases**

Press releases have been studied in the science communication (SCO) literature (Autzen, 2014), and recently as a source of news mentions (Ortega, 2020, 2021). However, there are no studies focused on the analysis of press releases as media objects with analytical value, and conceptually framed as spaces of interaction between media and science (Wouters et al., 2019; Costas et al., 2021). The main objective of this work is to fill this gap by illustrating how press releases (as particular SCO objects) can be quantitatively analyzed by applying similar tools and approaches as those applied in Scientometrics (SCI), using a scientometricinspired framework based on the existence of spaces of interactions of objects, actors, and impacts.

In Table 1 we illustrate how the dimensions used to study SCI can to a large extent be related to SCO. In both cases there are objects that are being generated (e.g., journal articles, books or conference proceedings in SCI, and press releases, blogposts, or news items in SCO). There are also actors involved in the development of these objects (e.g., research authors or scientific journals in SCI, and science journalists or bloggers in SCO). Finally, there are also different impacts that can be quantitatively captured for both SCI and SCO. While scientometrics capture impacts between scientific articles (e.g., citation linkages among articles), there are also different types of impacts derived from SCO objects (e.g., press releases mentioning other press releases).

Finally, scientific actors and non-scientific actors can interplay both in SCI (e.g., mentions of tweets to scientific publications) and SCO (e.g., mentions of tweets to press releases). Precisely, based on the notion of "heterogeneous cou-

#### Enrique Orduña-Malea and Rodrigo Costas

pling" (Costas et al., 2021), it is theoretically possible to frame and capture interactions between SCI and SCO objects, actors and impacts. In the context of press releases, it can be argued that when a press release directly links to (or mentions) a scientific article, this represents a coupling between the two realms of SCO and SCI. The underlying idea is that these forms of heterogeneous couplings are of fundamental relevance, since they can unveil dynamics and interactions between these two realms, which could be accommodated in the scientometric-inspired framework proposed in Table 1.


**Table 1:** Scientometric-inspired framework to measure science communication.

To discuss the potential of the scientometric-inspired framework described above, EurekAlert!, as the most comprehensive science press release distribution platform (Stockton, 2016), is taken as an illustrative case. First, a descriptive analysis is performed to present the main characteristics of press releases as SCO media objects. Furthermore, the online impact of press releases is studied through two online communication channels (the web at large and Twitter). Finally, the journals and publications mentioned in the EurekAlert! press releases (interactions) are also analyzed in order to determine which science is communicated by press releases, providing a basis for the further study of SCO-SCI heterogeneous couplings.

### **Methods**

#### **. Press releases data**

All press releases published by EurekAlert! since its inception were collected. Despite the existence of advance search features at the EurekAlert! website5 , the website does not support systematic large information retrieval options. As all press releases are published online as webpages under a specific URL address6 , it was decided to download all these documents directly via web crawling7 . To do this, SocSciBot8 v4 was configured to crawl all URLs under the "EurekAlert.org/press\_release/" fold (ethical guidelines were followed by notifying the webmaster about the process, and by establishing one query per second to avoid crawling overload). This process was carried out during the first week of March 2021, obtaining a html copy of each webpage published until 28 February 2021. At the end of the process, 456,758 individual files were downloaded. A data cleansing process aimed at filtering out all those files not corresponding to a press release (e.g., sitemaps, forms, automatic server messages) was carried out, which yielded a final set of 455,703 press releases. The html file of each press release included all descriptive metadata fields that were created and

 **5** https://www.eurekalert.org/search.php

**<sup>6</sup>** https://www.eurekalert.org/pub\_releases/\*

**<sup>7</sup>** While Eurekalert! provides a service for multilanguage press releases, covering Chinese, French, German, Japanese, Portuguese, and Spanish, the general URL of these press releases corresponds to the English version. This way, all online metrics gathered in the study are considering all language versions of each press release.

**<sup>8</sup>** http://socscibot.wlv.ac.uk

curated by the EurekAlert! staff. A python script (see supplementary material) was subsequently written to extract the following metadata: keywords, description, date, funder, journal, type, institution, meeting, and region.

This python script was also used to extract all DOIs mentioned in each press release to characterize the publications covered by press releases, not included in the metadata fields. Several errors with DOIs were found (e.g., broken URLs, shortened URLs). These errors were systematically found for some journals (e.g., APL Photonics), which were corrected whenever possible. After this curation process, a total of 99,829 unique DOIs were found in 98,305 press releases (21.6% of all press releases), a percentage close to that found by Bowman and Hassan (2019), who also reported a small share (18%) of EurekAlert! press releases including DOIs.

Finally, data linking to the EurekAlert! press releases was gathered from Twitter (1,364,563 mentioning tweets) and the web at large via Majestic (54,089,233 linking webpages). A detailed description of the collection procedure is given below.

#### **. Twitter data**

The Twitter API v2 was used to retrieve all tweets containing a URL leading to a press release published by EurekAlert! from 26 March 2006 (the day Twitter was launched) until 28 February 2021. To do this, the full-archive search endpoint, available through the Academic Research Product track, was used9 .

Despite some applications, such as Academic Mozdeh10, already operating with this endpoint, these tools offer a predefined set of parameters out of all those available in the API. For these reasons, an ad hoc python script was written to query the Twitter full-archive search endpoint directly, using the following search query: url:"EurekAlert.org/press\_release" -is:retweet

The full query (including dates of tweets publications, tweet-level metrics, and creator-level metrics) was debugged with Postman11, and the script was launched in 15 April 2021 via a local server located at the university where the first author is affiliated. All data was obtained in a json file, which was subsequently distilled with OpenRefine12 and finally exported to a spreadsheet to be

 **9** https://developer.twitter.com/en/solutions/academic-research

**<sup>10</sup>** http://mozdeh.wlv.ac.uk/AcademicResearchTwitter.htm

**<sup>11</sup>** https://www.postman.com

**<sup>12</sup>** https://openrefine.org

statistically analyzed. This process gathered a total of 1,496,125 original tweets (retweets were excluded due to the computational complexity involved in their analysis) with at least one URL referring to a EurekAlert! press release. After a subsequent cleansing task, a total of 1,364,563 tweets including at least a URL pointing to each EurekAlert! press releases were obtained, which form the final Twitter dataset.

All links submitted within tweets are eventually wrapped with the t.co shortener13. These links are automatically un-shortened in the API response. However, when the link embedded in the tweet is already shortened (e.g. embedding a bit.ly URL in a tweet), the API response does not completely unshorten this URL (i.e., the API just unshortens from t.co to bit.ly). To solve this problem, another python script was developed to un-shorten all unresolved short URLs. After this process, a total of 260,780 unique press releases URLs were finally obtained.

#### **. Web data**

Majestic Pro14 was used to discover links referring to EurekAlert! press releases in the web at large. The historic index (which covers URLs crawled from 1 September 2015 to 1 April 2021 at the time of writing this manuscript) was used15. This link intelligence tool has been successfully used in the literature for webometrics analyses (Orduña-Malea, 2021).

Using "EurekAlert.org/press\_releases/" as a seed URL path, the number of mentioning webpages (those webpages including at least one hyperlink to one specific URL under the URL seed path) and the number of mentioning websites (those websites including at least one hyperlink referring to one specific URL under the URL seed path) were gathered. As Majestic treats "http" and "https" URLs separately16, a merging process was carried out to obtain all link-related metrics for each press release regardless of the protocol used.

Finally, Majestic's Flow Metrics (Citation Flow and Trust Flow) were obtained for each URL. These flow metrics are meant to capture some notion of the

independent metrics.

 **13** https://developer.twitter.com/en/docs/tco

**<sup>14</sup>** https://majestic.com

**<sup>15</sup>** Majestic offers two URL indexes, the Fresh index (URLs found during the last 120 days) and a historic index, a huge database covering the last five years.

**<sup>16</sup>** https://www.eurekalert.org/pub\_releases/2015-12/aabu-paa122815.php and http://www. eurekalert.org/pub\_releases/2015-12/aabu-paa122815.php are formally different URLs with

"prestige" or reputation of each URL through their linking webpages (Jones, 2012). Citation Flow is a score on a scale between 0 and 100 achieved by one website, based on the number of hyperlinks it receives. It measures how often a URL is linked. Therefore, it measures the quantity of links received. Trust Flow is a score on a scale between 0 and 100 achieved by one URL. It is based on the number of hyperlinks (and clicks on these links) from trusted seed sites that the URL receives. Therefore, it measures authority and ability to generate web traffic17. All data was extracted directly via the Majestic Pro interface as of 9 May 2021 and exported to a spreadsheet for statistical analysis. The overall process is summarized in Figure 1.

**Figure 1:** Diagram of the overall process (sources, data collection, and stats).

 **17** The incorporation of these metrics only plays a role to illustrate the relevance of characterizing linking websites by their "prestige," but this does not represent a validation of this metric (which at best must happen in future research) nor a recommendation to be incorporated as a fixed element of the analytical framework proposed.

### **Results**

#### **. The objects: EurekAlert! press releases**

#### **.. How many press releases are in EurekAlert!?**

EurekAlert! has published 455,703 press releases online from its inception in 1996 until 28 February 202118. This number has grown continuously, achieving a milestone of 35,232 press releases published in a single year in 2020 (Figure 2). This growth is evidenced by the fact that the publication output from 2016 to 2020 represents 30% of the total number of press releases published to date.

**Figure 2:** Annual number of press releases published in EurekAlert! (1996 to 2000).

#### **.. Types of press releases in EurekAlert!**

Most press releases are those of the research type (82.6%). The presence of business and grant announcement press releases (4.9% each) is also remarka-

 **18** The crawling process discovered 10 press releases with an older publication date (both the URL and the documents publicly display 1969, while the html metadata of these documents display 1970). The authors associate these publication dates with human error.

ble, while the remaining categories are relatively infrequent (Table 2). The number of press releases published per day is not stable, the maximum having been detected on October 3 2016, when 228 press releases were published19. Autzen (2014) already detected a peak of press releases in 2013. Although this issue was referred to as exceptional, data shows the increasingly growth since then. The number of press releases mentioning a DOI is also displayed, evidencing an increasing growth. A total of 66.2% of all press releases published in 2020 included at least one DOI.


**Table 2:** Number of EurekAlert! press releases per publication type.

Note: 455,702 press releases with information in the "type" metadata field.

#### **.. What is the thematic distribution of EurekAlert! press releases?**

The predominance of medicine and health in the institutions submitting press releases is confirmed when analyzing the most frequently used terms included in the keywords field of each press release (Table 3). The term "medicine/health" appears in 41.2% of all press releases published.

 **19** That day Yoshinori Ohsumi won the Nobel Prize for Medicine, but no other singular event has been identified.


**Table 3:** Keywords most frequently used by EurekAlert! press releases.

N = 455,564 press releases with information in the "description" metadata field.

The high predominance of medicine-related keywords may explain the limited vocabulary employed by EurekAlert! to describe press releases, as only 254 different keywords have been detected. The co-occurrence of all these keywords represents the scientific topics covered by EurekAlert! press releases over the years and their relations (Figure 3). A first cluster (green) represents medicine and health; a second cluster (red) represents social sciences; a third cluster (yellow) represents biology and ecology; a fourth cluster (blue) represents engineering; and a fifth cluster (purple) represents a mixture of physics, climate change, and oceanography. As expected, "medicine/health" is the node with the highest link strength (742,688). Other strongly connected terms are biology 457,648), computer science (226,258), and public health (225,447).

The collection of medicine-related items allows tailored thematic analysis using press releases as SCO object. Appendix B includes an illustrative case study of press releases mentioning "covid-19" or "coronavirus," showing how the application of the analytics can be expanded to include more specific and topical perspectives than the ones presented here.

**Figure 3:** Map of co-occurrence of keywords in EurekAlert! press releases. Source: EurekAlert! data powered with VOSviewer (www.vosviewer.com). Map available online at: https://app.vosviewer.com/?json=https://drive.google.com/uc?id=1XhbI8m0lo8i5Ld6 SVrQXqD-dgK9wj1w5. N = 455,564 press releases with information in the "description" metadata field.

#### **. The actors: EurekAlert! press releases producers**

#### **.. Who are the most important producers of EurekAlert! press releases?**

The Public Information Officers (PIOs) submitting press releases to the EurekAlert! are the most important actors in the generation of press releases. The origin of PIOs is strongly dominated by North American institutions (72.8% of all press releases submitted), followed by press releases submitted from European PIOs (21.8%). The presence of both African (0.7%) and especially South American (0.5%) institutions is marginal (Figure 4). There is a rather obvious distribution bias towards North American institutions, which can be explained by the fact that only institutions affiliated to EurekAlert! are eligible to submit press releases, and these mostly come from North American academic institutions.

**Figure 4:** Number of press releases published in EurekAlert! per origin of PIOs. Note 1: 451,711 press releases with information in the "region" metadata field; note 2: y-axis in logarithmic scale.

The JAMA Network is found to be the institution with the greatest number of press releases submitted to EurekAlert! (7,333 press releases) followed by Goddard Space Flight Center–NASA space research laboratory in the United States (6,820), and the University of Texas (6,383). The results obtained are close to those obtained by Bowman and Hassan (2019). Supplementary material (Appendix C) includes the most active PIOs submitting press releases, organized into journals and publishers, associations and federations, universities and other institutions (e.g., hospitals not affiliated to universities, national institutes and government bodies). Despite associations acting as journal publishers, these entities have been categorized independently for the sake of clarity. Only in those cases when the institution marked as PIO by EurekAlert! corresponds to a specific entity's publication (e.g., Proceedings of the National Academy of Sciences) has it been included as a journal. In the case of universities, all internal units (e.g., medical centers, university presses, or research institutes) have been merged to obtain a unique value for universities as a type of institution.

These results show a remarkable presence of medical and health related journals and associations. Likewise, medical centers (e.g., University of Texas Health Science Center at Houston, Georgetown University Medical Center or Columbia University Medical Center) and schools (e.g., Johns Hopkins Medicine, Boston University School of Medicine or Michigan Medicine) constitute the most active EurekAlert! PIOs within universities.

#### **. The impact: mentions to EurekAlert! press releases**

#### **.. What is the online impact of EurekAlert! press releases?**

Tweets and websites linking to EurekAlert! press releases represent forms of impact related to the press releases themselves (see Table 1). The online impact of press releases on social media has been measured via Twitter. A total of 1,364,563 tweets with at least a URL linking to a EurekAlert! press release have been analyzed. The number of mentioning tweets increases especially from 2010 onwards20 and reaches a maximum by 2016 (238,881 tweets mentioning EurekAlert! press releases) (Figure 5).

Since 2016, the number of mentioning tweets has notably decreased, as well as the annual average of tweets per press release. A detailed analysis of the Twitter users linking to EurekAlert! press releases is available in the supplementary material (Appendix D). The online impact of press releases on the web at large has been measured via Majestic, through the 54,089,233 webpages with at least a URL linking to a EurekAlert! press release. The maximum value is observed in 2020, with 9,311,788 different mentioning webpages linking to press releases (Figure 6). The supplementary material (Appendix E) includes detailed information related to those domain names providing the most links to the EurekAlert! press releases.

 **20** This effect is attributed to the creation of the EurekAlert! official Twitter account in September 2009.

**Figure 5:** Number of tweets including a link to EurekAlert! press release per year. Note: tweets per press release considers tweets and press releases both published the same year. Note: yaxis (left) in logarithmic scale.

The analysis of all mentioning URLs extracted from Twitter and Majestic has revealed the existence of a sort of obsolescence effect, that is, the existence of active URLs not referring to press releases (i.e., outdated URLs). This effect is attributed to the fact that there are press releases that have been moved to a new URL or have been removed, while the old URL is still active, but referring to other contents21. This analysis has detected 907 outdated URLs in tweets and 2,039 outdated URLs in webpages.

The number of unique press releases being mentioned (either from Twitter or webpages) per press release publication year is offered in Table 4, excluding all outdated URLs. As we can observe, the coverage on Twitter is low until 2010- 2011 (mainly attributed to the activity of the EurekAlert! official Twitter account [@EurekAlert], created in September 2009). Overall, 56.7% of all press releases have been mentioned at least once on Twitter22. From 2016-2017 onwards a decreasing trend is observed, which is in line with results previously displayed in Figure 5. The coverage of press releases on the web at large is otherwise significantly greater (79.1%).

 **21** For example: https://www.eurekalert.org/pub\_releases/2000-10/ASfM-Natd-2910100.php

**<sup>22</sup>** Twitter was launched in 2006. Therefore, it cannot be expected that many tweets will link to press releases published before this year.

**Figure 6:** Number of webpages including a link to EurekAlert! press releases per year. Note: yaxis in logarithmic scale.


**Table 4:** Percentage of EurekAlert! press releases tweeted and web-linked (1996-2020).


#### **. Heterogeneous couplings: press releases mentioning scientific articles**

#### **.. Which journals are covered in EurekAlert!?**

The sources reported in EurekAlert! press releases can be seen as heterogeneous interactions between SCI objects (i.e., the journal) and SCO objects (i.e., the press release). A total of 12,071 unique scientific sources (including journals and conference proceedings) have been identified. The results obtained are also close to those obtained by Bowman and Hassan (2019), with the novelty of the increasing presence of the journal *Scientific Reports*. Table 5 includes the number of press releases where each publication source appears. In addition, we provide the number of publications with DOI published by each journal in the same period (1996 to 2020) according to Scopus. This way we can estimate what percentage of publications published by each scientific journal has been covered by EurekAlert! press releases.

Taking aside multidisciplinary journals (*PNAS, Science, and Nature*), a heavy representation of medicine and health journals (e.g., *British Medical Journal, Cell, JAMA, Lancet, New England Journal of Medicine* or *PLoS Medicine*) is found, which again confirms the propensity of covered research in this field on EurekAlert!. The significant percentage of publications from *PLoS Medicine*

(47.9% of publications in the period covered) and Science Advances (43% of publications) is also noteworthy.23


**Table 5:** Number of EurekAlert! press releases per publication type.

N = 309,196 press releases with journal information.

 **23** These percentages are even more significant if we consider that *PLoS Medicine* began operation in 2004 and *Scientific Advances* in 2015.

### **Discussion**

This study illustrates how press releases can be quantitatively analyzed, applying similar tools and approaches as those applied in scientometric research. In this regard, the consideration of press releases as analytical objects with applicability to measure science communication interactions is discussed.

Press releases act as a filter of science by mentioning and promoting specific publications. As such we argue that press releases work as spaces of interaction between science and science communication. We assume that the selected publications might have specific value attributes, either academically oriented (i.e., new discoveries with great implications for research) and/or mediaoriented (e.g., controversial results, topics of public interest). In either case, the study of these objects allows us to obtain a better understanding of science communication mechanisms due to the interactions established by press releases with other objects (scientific publications, journals, tweets, websites) and actors (PIOs, tweeters, and website authors). These types of interactions have not been measured so far. As the number of published press releases is growing (more than 30,000 items published annually by EurekAlert! alone), there arguably is a critical mass that permits large scale analyses, and which enables the introduction of advanced quantitative approaches to study SCI-SCO dynamics.

As the creation and dissemination processes of press releases follow different dynamics and purposes than scientific publications, the consideration of press releases as independent media objects with analytical value raises a series of peculiarities to consider. Below we discuss the most important data limitations of EurekAlert! as well as the main types of interactions captured through the proposed framework. We acknowledge that this is a first proposal of how to study press releases as science communication objects. Future research should focus on further developing this framework and its analytical scope.

#### **. EurekAlert! data limitations**

Press releases metadata fields have remained the same since the launch of EurekAlert!, allowing the realization of quantitative analyses for the entire period of existence of EurekAlert!. However, most of the metadata fields are not harmonized. Consequently, institution and journal names appear under different variants, or typographic errors, which limit the quantitative analysis of the data without a substantial investment in data curation. The thematic keywords describing the scientific topics of press releases is one of the few controlled data elements in EurekAlert!. However, the vocabulary consists of only 254 keywords, which makes this controlled set of keywords somehow limited, particularly when describing new topics (e.g. COVID-19) or smaller fields. The expansion of the classification scheme used to describe press releases (e.g. including article-level data or expanding the keywords used) would provide EurekAlert! with a more dynamic and valuable tool to identify (and study) the topics of the press releases.

#### **. Objects, actors, and interactions**

This study frames the investigation of press releases as spaces of interaction between science (SCI) and science communication (SCO). This could be seen as a derivation of the "heterogeneous couplings" framework proposed by Costas et al. (2020). As such, the framework proposed in this study considers press releases as SCO objects, produced by different SCO actors (e.g., PIOs, journalists), and receiving impact (e.g., tweets, links). Each SCO element (objects, actors, impacts) can in turn interact with other SCO and SCI elements. In this study we have illustrated how some of these objects, actors, and impacts related to SCO and SCI can be quantitatively captured and combined.

#### **. Objects: press releases as spaces of SCI-SCO interaction**

Press releases can be seen as objects in the science communication process. At the same time, quite often, press releases interact with science objects (e.g., by linking directly to scientific publications). In some cases, these publications are linked as supplementary readings, but in other cases the press release is actively promoting these publications. This clearly illustrates the role of press releases as spaces of interaction between science communication (SCO) and science (SCI). In this study, we have illustrated firstly, how scientific publications are mentioned in press releases and secondly, how their own features (e.g., their journals of publication) can be further studied.

Only 21.6% of all EurekAlert! press releases include a link to at least one DOI. This low percentage is mainly attributed to the fact that EurekAlert! was launched long before the introduction of DOIs as the main standard to identify scientific publications. In those early years, publications were mentioned by URLs to journal websites without a DOI, or just via textual mention. The situation has changed over time, with more than 66% of EurekAlert! press releases mentioning a DOI in 2020. The increasing mention of DOIs in press releases

opens up the possibility for more ambitious studies of SCO-SCI interactions, wherein features of press releases (e.g., content, producers, or received impact) can be studied in the context of the features of the scientific publications mentioned in the press release (e.g., scientific authors, scientific journal, covered topics, etc.).

#### **. Actors: press releases producers**

Public Information Officers (PIOs) registered in EurekAlert! are typically the press release producers. This field may correspond to the journal or publisher where the article was published or the affiliation of one author of the scientific publication covered in that press release (generally but not necessarily the corresponding author). By measuring these actors, we can capture the activity of communication offices in promoting research. The specific publishers' active policies related to press releases submissions (and their investment in the operating of press offices) might introduce an inherent bias that should be considered.

The fact that large institutions may have different press offices operating makes institution-level analyses difficult. The existence of press offices for schools, departments or research centers may hinder the presence of the universities, while the existence of press offices for large editorials (e.g., Wiley or Elsevier) may hinder the presence of specific journals as PIOs.

EurekAlert! press releases show a strong North American bias in the coverage of science news. Like in the field of scientometrics when discussing the coverage and biases of its data sources (Martín-Martín et al., 2021; Visser et al., 2021), the coverage of press release sources (e.g., in the future also including AlphaGalileo and other science news platforms like The Conversation [Dudek & Costas, 2020]) can be seen as an additional important step in the development of quantitative studies of science communication.

#### **. Impact: mentions to press releases**

Like scientific publications, which in scientometric approaches can be measured in terms of their impact either within science (e.g., citations among scientific publications) or outside of science (e.g., altmetrics), press releases may also generate further impact of their own. The existence of a distinctive URL for each press release allows this object to be used to measure impact-related events from which to generate a set of online metrics at the press release level. In this case, the mention of URLs both in websites and tweets has been studied.

With regard to Twitter, a very large set of tweets with at least one URL referring to a press release has been collected (1,364,563 tweets). However, only around half (56.7%) of all press releases have been tweeted at least once, and the percentage of annual tweets per press release has been decreasing since 2016, which might indicate a decline of Twitter as a communication channel for EurekAlert! press releases. In the case of the web at large, the coverage is much broader (79.1% of press releases have been linked at least once), receiving links from 54,089,233 webpages, mainly from organizations (.org top-level domain names), academic-related websites, and US universities.

Other forms of online impact (e.g., downloads, views, republication in other media) could also be seen as additional forms of the impact of press releases to be further investigated.

### **Conclusions**

This study introduces a scientometric-inspired framework for the quantitative study of press releases as a novel information source for the study of SCO objects and their interactions with SCI elements. The large volume of press releases published and their wide online dissemination make these objects relevant in the measurement of SCO-SCI interactions.

Future research is recommended regarding the expansion of data sources related to SCO objects by considering other national and international press releases platforms, as well as monitoring other online channels of SCO dissemination (e.g., The Conversation). The ultimate ambition of the development of quantitative studies of press releases (and other SCO objects) is to gain a better understanding of SCI and SCO interactions. This understanding will enable the exploration of more ambitious questions regarding the communication of science and its effects on measuring the societal impact of SCI objects within the context of new emerging big data quantitative approaches, many of which have historically already been used to study science dynamics, such as scientometrics, webometrics and altmetrics.

### **Supplementary material**

A dataset including scripts, raw data, and supplementary material is available at the following URL: https://riunet.upv.es/handle/10251/186769

### **References**


Jones, D. (2012). Flow Metrics™ will change the way you look at links. *Majestic Blog*. https://blog.majestic.com/development/flow-metrics

	- https://doi.org/10.1177/107554709701800400

Lemke, S., Brede, M., Rotgeri, S., & Peters, I. (2022). Research articles promoted in embargo emails receive higher citations and altmetrics. *Scientometrics*. https://doi.org/10.1007/s11192-021-04217-1


Orduña-Malea, E. (2021). Dot-science top level domain: Academic websites or dumpsites? *Scientometrics, 126*(4), 3565-3591. https://doi.org/10.1007/s11192-020-03832-8


Weingart, P. (2012). The lure of the mass media and its repercussions on science. In S. Rödder, M. Franzen, & P. Weingart (Eds.), *The sciences' media connection–Public communication and its repercussions* (pp. 17-32). Springer. https://doi.org/10.1007/978- 94-007-2085- 5\_2


#### Irene Broer

## **2. Curating, transforming, constructing science news: The newsmaking routines of Science Media Center Germany**

**Abstract:** Science Media Center (SMC) Germany contributes to the construction of science news with routines of curation, selection, framing, and broadcasting that bridge the norms of science and journalism. These include restricting the scope of scientific topics and sources; assigning scientific topics with contextual scientific, social, and journalistic relevance criteria; enhancing content with external expertise; and timing broadcasts with the intention of promoting, altering, or preventing coverage of science issues. In collective sense-making processes, editors negotiate both explicit and implicit knowledge about science and journalism, informed by input from scientific actors invested with explanatory power. Based on criteria of knowledge production from both science and journalism, these routines grant epistemic authority to the science content curated by SMC Germany.

**Keywords:** science journalism, science media center, newsroom ethnography, editorial process, COVID-19

### **Introduction**

Science Media Centers (SMCs) are organizations that bridge the fields of science and news media, providing journalists with curated and freely accessible science content in the form of press releases, press briefings, and expert statements (Broer & Pröschel, 2021; Fox, 2012; Rödder, 2020). While scholarly interest in SMCs is rising (Broer & Pröschel, 2022; Buschow et al., 2022; Suhr et al., 2022), there is still a lack of insight into their newsmaking routines. Drawing on findings from an ethnographic newsroom study, this chapter explores the routines through which SMC Germany contributes to the construction of science news. I consider science news the result of a type of knowledge production in which scientific knowledge and expertise is located, filtered, and transformed to fit the norms of journalistic reporting. Following the literature on the epistemology of journalism (Ekström & Westlund, 2019; Ettema & Glasser, 1984; Matheson &

Wahl-Jorgensen, 2020), I am interested in the "*rules*, *routines* and *institutionalized procedures*" by which SMC Germany produces knowledge by curating scientific topics and expertise for journalism (Ekström, 2002, p. 260, original italics). Below, I first explore the rise of intermediaries in the production of science news before introducing SMC Germany as a case study. After outlining my methodology, I structure the routines through which SMC Germany contributes to the production of science news along five phases of news production: access and observation, selection and filtering, editing and processing, distribution, interpretation and feedback (Tandoc & Duffy, 2019). Throughout, particular attention is paid to the ways in which editors negotiate routines to overcome practical and epistemic challenges.

### **Third parties in the production of science news**

As a result of technological and societal changes, the communication landscape between science, media, and the public has become more diverse in terms of actors, practices, and orientations (Broer & Hasebrink, 2022; Bucchi & Trench, 2021; Neuberger et al., 2019). The neoliberalization of scientific research has led to a rise in strategic science communication and a struggle for visibility (Raupp, 2017; Väliverronen, 2021). This trend is reflected in increased communication efforts by research institutions (Autzen, 2014; Friedrichsmeier et al., 2015), academic publishers (Franzen, 2012), and individual scientists (Peters, 2021). Meanwhile, professional science journalism, like other forms of journalism, has come under pressure. The development of digital media has led to losses in advertising revenue and subscriptions for traditional media outlets, resulting in the downsizing of newsrooms and an overall decline in professional science journalists (Dunwoody, 2021; Schäfer, 2017). The demands of the 24-7 news cycle and the rise of social media have placed additional pressure on science journalists to produce more content quicker, making in-depth coverage of scientific issues more challenging (Ashwell, 2016). This has led to alternative solutions (Carlson & Usher, 2016; Hepp & Loosen, 2022; Suhr et al., 2022) such as new types of organizations that outsource parts of news work to actors outside of traditional newsrooms. In science communication, this development manifests itself in the form of intermediary organizations which consciously position themselves in the gray areas of science communication (Gerber et al., 2020, p. 50; Görke & Rhomberg, 2017, p. 54).

Third parties between science and media have a long history in science communication (Moles et al., 1967). One example is the Science Service, a news syndicate founded in the United States in 1921 with the aim of translating and interpreting science for journalists (Nelkin, 1987, p. 87; Ritter, 1926, p. 200). A century later, Guenther and Joubert (2021) note the emergence of "novel interfaces" that use digital media to enhance the dissemination of scientific knowledge through the news media. The Conversation, for example, provides editorial support for scientists to write articles about their expertise (Guenther & Joubert, 2021), while SciDev.Net provides free reporting on science and technology with the goal of aiding sustainable development and poverty reduction (Trench, 2008), and SMCs curate scientific developments and expert statements for journalistic use (Broer, 2020; Rödder, 2020). All of these intermediaries share the common goal of increasing public attention to scientific issues with the aim of improving individual decision-making. In doing so, they aim to support the norms of liberal democracy and its principles of informed opinion-forming and decision-making (Nowotny et al., 2001). But they also challenge traditional roles, norms, and practices in journalism (Guenther & Joubert, 2021; Williams & Gajevic, 2013). Understanding how intermediary organizations produce science news is essential to understanding their potential implications for the future of science communication and journalism.

### **Case study: Science Media Center Germany**

The non-profit organization Science Media Center (SMC) Germany, based in Cologne, is one of seven independently operating SMCs worldwide that aim to "inform public debate and discussion on the major issues of the day by injecting evidence-based science into headline news" (SMC U.K., 2012). To achieve this, these organizations provide registered journalists with free-to-use summaries of scientific research, and expert assessments of new scientific claims and current science topics in the public debate. The concept was born in the United Kingdom where the first SMC was established in 2002. Since then, others were founded (in chronological order) in Canada, Australia, New Zealand, Japan, Taiwan, Germany, Kenya and Spain. At the time of writing, however, the SMCs in Canada and Japan appear to be inactive. While the goals and missions of the international SMCs are similar, the degree to which they are aligned with either science or journalism varies. Whereas SMC UK was established as a "press office for science" (Fox, 2012, p. 257), SMC Germany was founded by members of the German professional association of science journalists (Wissenschafts-Pressekonferenz e.V.)

with the aim of supporting journalists in their work by making scientific research more accessible (Hettwer et al., 2013).

In 2020, SMC Germany had three pillars: the editorial department, which consisted of eight full-time editors working in the sections: (1) Health & Medicine, (2) Technology, Energy, Mobility & Artificial Intelligence, and (3) Climate & Environment. Second is the SMC Lab, which develops a variety of software to assist the editorial department. Third is the Innovation Digital Media (IDM) department, which was set up to support the development of new solutions at the interface between the editorial department and the SMC Lab. The Klaus Tschira Foundation (KTS) is the primary sponsor of SMC Germany. The organization also receives financial support from grants and donations. The organization's editorial autonomy is stipulated in its corporal charter (SMC Germany, 2021). During the time of fieldwork in 2020, SMC Germany had 800 journalists registered to receive content; at the time of writing in 2023, this number had risen to 1900 journalists.

Empirical research on SMCs is limited, although there has been increasing interest. There is some evidence that SMCs can influence how contentious science issues are presented in the media. Williams and Gajevic, for example, showed that SMC UK led a successful communication campaign in favor of more lenient regulations on research involving animal-human hybrid embryos (2013). Recent studies of SMC Germany have focused on its role in field repair and field enhancement for journalism (Buschow et al., 2022), as well as its role in science communication as a knowledge broker, trust broker, and value broker (Broer & Pröschel, 2022). Suhr and colleagues (2022) conclude that the organization type of SMC Germany, due to its particular financial setup and the structure of science journalism, cannot directly be transposed to other intersections between journalism and fields of expertise, e.g., in political or business reporting.

Despite this recent interest, however, there is still a lack of understanding of the internal workings of SMCs and the routines with which they contribute to the construction of science news (Rödder 2020). In order to fill this gap, I sought to uncover the editorial routines through which SMC Germany selects, transforms, and mediates scientific expertise to its journalistic audience. In the following paragraphs, I answer the following research question: through which routines does SMC Germany contribute to the production of science news?

### **Methodology**

The insights on the routines of SMC Germany are based on ethnographic material. Newsroom ethnographies have a long tradition in journalism research, as seen in the works of Fishman (1980), Schudson (1989), and Tuchman (1980). They allow researchers to open the "black box" (Stonbely, 2015, p. 260) of news production, as the situated practices that shape news remain a blind spot in studies using indirect data from retrospective interviews or content analyses (Ryfe, 2018). This is increasingly important given the rapidly changing journalistic environment (Cottle, 2009; Domingo, 2011; Westlund & Ekström, 2020).

During the first phase of fieldwork, I was present at SMC Germany from January 6 to 31, 2020, and participated in editorial and management meetings, informal conversations with staff, and had access to the organization's collaboration software, Slack. Despite minor language barriers, I was able to gain a firsthand understanding of SMC Germany's day-to-day operations. Due to the unexpected outbreak of the novel coronavirus in January 2020, the impact of the COVID-19 pandemic on SMC Germany's routines also became a focus of the study. The second phase of fieldwork, conducted from October 5 to 30, 2020, allowed me to deepen previous insights and assess changes in practices, routines, and role understandings since the start of the pandemic. This phase of fieldwork consisted of both in-person and virtual components.

The final ethnographic material consisted of 28 semi-structured and unstructured interviews with all editorial staff, 42 field notes of editorial meetings, 168 SMC Germany broadcasts, and 62 chat logs. These were analyzed using the constructivist grounded theory approach (Charmaz, 2006: 23). The ethnographic material was analyzed using MAXQDA software. I began with open, line-by-line coding and focused coding to identify initial topics. Then I proceeded to axial coding to connect and form abstract codes.

### **Routines of SMC Germany**

If news work is a construction of reality, routines are the situated practices, rules, and procedures through which this takes place (Ekström, 2002, p. 260; Molotch & Lester, 1974, p. 105). Although conceptualizations tend to focus on the ways that routines are shaped either by organizational contexts (Altmeppen, 2006, 2008) or by social practices (Ryfe, 2016, 2018) they typically come into being through interactions between both (Westlund & Ekström, 2020, p. 73). In the following, I structure SMC Germany's routines along five stages of news production (Tandoc & Duffy, 2019): access and observation; selection and filtering; editing and processing; distribution; and interpretation and feedback.

#### **. Access and observation**

Keeping an overview of scientific developments and the public debate on science is an important and challenging part of news work at SMC Germany. The organization has limited its key interests to research on medical, environmental, and technological topics, particularly those relating to mobility, energy, and artificial intelligence. These topics correspond to SMC Germany's three editorial departments and broadly reflect the top science issues covered by German media (Badenschier & Wormer, 2012). The division of its newsroom into topic-specific departments is akin to the "beat system" in traditional media (Fishman, 1980). The editors are expected to apply their so-called explicit knowledge (Grant, 1996; Patterson, 2013) to judge the significance of developments in distinct scientific subfields.

In addition, SMC Germany has set up editorial and automated sourcing practices. The main sources for new scientific findings are embargo e-mails from scientific journals and press releases from research institutions. The embargo system is widely used in science journalism (Hermida, 2010; Kiernan, 2006), and refers to the act of sending out announcements of upcoming scientific publications under a restrictive deadline. The embargo period should ensure that journalists have sufficient time to prepare for reporting on potentially complex topics (Franzen, 2014). In 2020, SMC Germany was subscribed to the embargo services of around 40 academic publishing journals, mostly belonging to larger publishing houses focused on medical, life, and natural sciences. These typically send out weekly embargo e-mails with the titles of upcoming scientific publications. Because the format of embargo e-mails varies widely, the in-house SMC Lab has developed software, internally referred to as "Newsdesk," that automatically scans and displays them in a standardized manner. Typically, the embargo period comprises seven days. SMC Germany editors use this time to select and report on new research.

Another source for new research are platforms like EurekAlert!, and to a lesser extent AlphaGalileo and the German-language Informationsdienst Wissenschaft. As explored in detail by Orduña-Malea and Costas (see Chapter 1 in this book), these platforms display press releases from paying organizations that conduct, publish or fund scientific research. During my fieldwork, SMC Germany editors did not have set routines for checking these platforms; instead, they browsed their webpages when they could afford to do so.

In addition to peer-reviewed publications, SMC Germany editors also use preprints, i.e., scientific texts awaiting a peer-reviewed publication process (Walker & Rocha da Silva, 2015), as a resource. At the start of the fieldwork, SMC Germany used preprints as an indicator of scientific issues that may become relevant in the

future. The SMC Lab has created a program that scans preprint databases for keywords relating to SMC Germany's main topics of interest in order to flag preprints with above-average download counts:

We wanted to see papers that we wouldn't see otherwise, but that could be relevant for the public, and there are distinct topics [that we look out for]. The idea was basically that interesting papers in research are perhaps marked, at least if they are very interesting and many people are interested in them, by the fact that they have a high download count early on. (Interview, editor-in-chief, January 2020)

Like in other science newsrooms (see Chapter 3 in this book, also Fleerackers et al., 2022), preprints became more relevant during the COVID-19 pandemic, as researchers began uploading new findings on the novel coronavirus on preprint databases before finishing the academic peer review and publication process (Fraser et al., 2020). Whilst this development allowed SMC Germany to stay on top of the latest insights, albeit unverified, it also meant that the organization could no longer rely on the time advantage of embargos, nor the preselection and quality assurance expected from peer-reviewed journals:

We had to change the way we did things [because] we didn't really stand a chance without an embargo. The corona studies were always released for direct publication. That is to say, they were sent out through the press offices, and of course they were immediately available to the journalists, who would write about them directly. So we just didn't have that kind of lead time anymore. (Interview, editor, October 2020)

SMC Germany also keeps a close eye on the public debate on science. To this end, an automated scan searches a press database with local, national, and international news media for keywords of the organization's main interests. Each morning, one of the SMC Germany editors takes on the rotating task of evaluating the results of the press scan. They summarize their findings, i.e., which media reported on a particular scientific issue, in a preformatted document that serves as the basis for the morning editorial meeting. Press monitoring helps editors keep track of scientific topics currently attracting media attention, which in turn informs decisions about timing potential broadcasts:

That's the core of it. We have built up a second observation radar. This is the so-called media monitoring. That means we look at what are public issues now and how do they develop over time? And when are the entry points for coverage where a scientific perspective can make a difference? (Interview, head of IDM department, January 2020)

Another source for observation are the press calendars of the German Press Agency (dpa) and those of German federal and local, and European political institutions. The former is monitored by means of a subscription service, whereas the latter are monitored manually by the editors themselves. These sources help the editorial staff to forecast which science and technology topics will be on the political agenda in the days to come, for example, as a result of new reports or policy advice documents being released. This particularly helps the editors working on climate and technology topics to gain an overview of their area of expertise, as the embargo e-mails and press releases tend to focus on research in medical and life sciences. Finally, SMC Germany is connected to other, international SMCs in the UK, Australia, and New Zealand via messenger tools.

#### **. Selection and filtering**

Like science journalists in traditional newsrooms, the staff of SMC Germany selects the topics for its broadcasts ("stories") according to several considerations. These journalistic, scientific, strategic, and organizational criteria are based on internalized notions about science and journalism (see e.g., Caple & Bednarek, 2016; Donsbach, 2004), expertise and explicit knowledge about the subject matter (Westlund & Ekström, 2020), and practical judgements based on SMC Germany's capabilities and agenda.

#### **.. Journalistic criteria**

Issue selection depends in part on news factors, i.e., those criteria with which journalists determine the news value of a topic prior to its selection (Harcup & O'Neill, 2017). Studies on issue selection in science journalism show that conventional news factors like actuality, unexpectedness, and range of impact apply to science coverage as well, albeit with some particularities (Badenschier & Wormer, 2012). The most important explicit criteria for topic selection at SMC Germany is that all "stories" must concern a so-called "public issue." Following observations of editorial meetings, chat discussions, and interviews with staff, determining whether a topic is a "public issue" involves a combined ascription of journalistic relevance, social relevance, and scientific relevance:

We don't do politics here and we're not an NGO or agency that passes on everything unfiltered, nor are we an agency *for* science, but we select what is relevant in the debate according to journalistic criteria, and who and what scientific expertise *should appear* in the debate. (Interview, head of IDM department, January 2020)

A topic is considered socially relevant when it has the potential to affect the lives of many people; when it is the current subject of public debate; when it is considered urgent; or in need of political decision-making. For new scientific findings, an important marker of social relevance is whether they are expected to have direct applicability:

[Editor 1] "Nature Catalysis published a study [Jie et al. 2020] about plastic waste which was treated with microwaves. They looked at what extensions and selectivity could be used to fragment this garbage and then convert it into hydrogen and fuels."

[Editor 2] "They only used small samples, though, so I'm not sure that would be a starting point right now. What's your impression?"

[Editor 3] "What does this fragmentation mean, are the parts then no longer harmful, do they stay around?"

[Deputy editor-in-chief] "Do we have any experts on this at all, dealing with exactly these chemical processes?"

[Editor 2] "I think it's interesting. I mean, there were these bacteria that ate a fingernailsized piece of plastic before, and that got a lot of media coverage at the time. But I don't know at what point these newer techniques are ready to be used."

[Deputy editor-in-chief] "You can ask what phase the technology is in, so whether it's already close to application or not at all yet."

(Excerpt field note, editorial meeting, 7 October 2020).

Journalistic relevance is assessed by means of media attention, whether already evident or expected in the future. The editors assess whether issues are perennial, recurring, or one-off events. To objectify this, SMC Germany editors sometimes link their hunches to topic careers: visualized media attention curves (Ruß-Mohl, 1993). SMC Germany employs these to choose the best moment to intervene, and maximize the impact of its broadcasts. Despite these explicit criteria and prediction routines, the decision to select an issue is still decided based on journalistic intuition or "gut feeling" (Schultz, 2007): a learning process that is linked to professional experience:

The biggest thing I've learned, and I would say I'm not done yet, is to make the decision: Is this an issue we should jump on or not? How relevant is it to society? How relevant is it in general? Or how interesting is it to me because *I* find it exciting? That's the hardest part. I mean, all the techniques, all the craftsmanship, you learn that. You can always learn the craft. *But developing a feel for issues is the biggest challenge, and it still is a little bit*. (Interview, editor, January 2020)

It is interesting to note that the editors of SMC Germany explicitly exclude the news factor "astonishment" from its "public issue" criteria, which seems to be important in regular science newsrooms (Badenschier & Wormer, 2012, p. 78). By excluding this news factor, SMC Germany aims to distinguish itself as an organization that provides quality, curated science news, while minimizing the possibility that its content will be seen as entertaining and frivolous ("unseriös").

#### **.. Scientific criteria**

To determine scientific relevance, the editors assess the novelty and credibility of scientific studies, as well as of scientific claims that are receiving attention in the public debate. To assess scientific novelty, the editors initially rely on their own explicit expertise:

I always think about what has the greatest informative value, what could you best write a story about, and how relevant it is. Sometimes, there are publications that highlight side issues. Or they only bring a smaller added value. Or they confirm existing knowledge or something like that. So for the fifth interesting cluster analysis, you kind of think ... We don't include every publication anymore. (Interview, editor, October 2020)

The editorial team assesses scientific credibility by looking at methodology, sample size, statistical analyses, and levels of evidence and verification. Here, double blind studies, a large sample size, statistical significance based on transparent inferences, meta reviews, and peer-reviewed publications are considered the most credible. This approach worked best for quantitative studies and medical research in particular. SMC Germany did not appear to have explicit criteria for determining the credibility of other types of research.

In addition, the editorial staff considers the academic standing of scientists who authored a publication or appeared as experts in the public debate, based on whether they have previously published on the subject matter. The perceived quality of the journal in which publications appear also matters. Studies published after peer review in international journals with high impact factors are considered most credible. It is interesting to witness that open access (OA) journals are perceived as less trustworthy than subscription-based journals:

There is the feeling, not with me, but with some in the editorial team, that [OA publications] are simply being pushed through and that there is no real peer-review process going on. [...] Peer review and openness are sometimes mixed up a bit, and some people just feel like this "open access" thing can't be good. It's too fast, that's why it's open. (Interview, editor, August 2020)

This belief is further fueled by the notion that scholars and research institutions could pay their way into OA publications. This seemed to conflate the phenomenon of predatory publishing (Grudniewicz et al., 2019) with the article processing charges that are common in OA publishing (Solomon & Björk, 2012). Being perceived as less trustworthy by science journalists may be an unexpected side effect of the shift towards open science in academia, which tends to be normatively linked to ideas of fairness, accessibility, and transparency (Taubert et al., 2019, pp. 5–7).

Finally, SMC Germany editors may select scientific issues because of a perceived lack of scientific credibility or quality. In case of doubt, the editorial staff will have the publication or its claims assessed by external experts and, should the concerns be justified, broadcast what the editors refer to as a "showstopper." SMC Germany hereby aims to prevent journalistic media picking up claims that are exaggerated, wrong or potentially harmful. This happened during the fieldwork in response to a new study published in Nature Energy (Sovacool et al., 2020), which examined the relationship between nuclear energy and carbon emissions:

[Editor 1] "We discussed this study at some length. [SMC Lab employee] explained to us that the researchers did not find any significant reduction. So as we feared, the Bayerischer Rundfunk, for example, is now already reporting that nuclear energy is therefore of no use. *So that has gone a little bit wrong, especially if it is going to run in the dpa* [German Press Agency]*. I'm unhappy with that.*"

[Deputy editor in chief] "Today is Tuesday, if *dpa* is planning something for Thursday, can we still do something, somehow get the study assessed by experts?"

[Editor 2] "We may not make it. We can at least contact dpa and describe our concerns and support them with statistical interpretation. That way we don't need to spam our journalists."

(Excerpt fieldnote, editorial meeting, 6 October 2020)

#### **.. Strategic criteria**

The organization's overall goals always weigh into SMC Germany topic selection process. An important criteria here is the extent to which a broadcast could make a difference ("einen Unterschied machen") in the public debate. During the fieldwork, this was discussed in relation to important upcoming negotiations in the European parliament about agricultural spending, which was missed due to staff absence:

[Editor 1] "This is about a really big chunk of money. [...] Actually, it would be our task to be aware of this four weeks in advance *in order to explain to the journalists where to look.* I don't see any concrete starting point for us now where scientists could contribute their expertise, the details of the negotiations are unclear so far."

[Editor-in-chief] "Exactly that would be a question for scientists [...] Greenpeace is making statements and also various other actors, but so far no scientists. Of course, there are also economic and ecological aspects. *We could shed light on all of that with science!* [...] The week is not over yet. It would be good to hear what the *voice, no*, *the voices of science* have to say about this."

[Editor 1] "I can think of some agricultural economists, I'll try to get people on the phone." [Editor-in-chief] "Yes. I see the problem that only lobbyists are heard, but the scientists are not. *We could change that, couldn't we?*"

(Excerpt field notes, editorial meeting, 21 October 2020)

#### **.. Organizational criteria**

Finally, if a topic meets the scientific, journalistic, and strategic criteria, the editorial staff examines whether reporting is feasible based on organizational capacity. The team's availability as well as the time window until broadcasting are both important considerations. Considering the small size of the editorial, limited availability due to illness, vacation or a high workload can have a strong impact on the issue selection process. The same applies to missing in-house expertise: topics outside of SMC Germany's thematic departments are unlikely to be reported on, despite gaining attention in media and society. Lastly, coverage may be canceled if an embargo period is too short, or if experts willing to provide statements or appear in a press briefing cannot be found in time.

#### **. Editing and Processing**

In the third stage, the selected issues are transformed into news through summarizing, contacting expert sources, and formatting. At SMC Germany, these routines tend to overlap, as considering the relevance, a good angle and hook, the right timing and the right format for a given scientific issue is done through "collective sense-making" between editors as well as expert sources, which imbues topics with meaning (Brüggemann, 2014, p. 64; Cook, 1998).

#### **.. Broadcasting formats**

In 2020, the organization had several broadcasting formats, including "Rapid Reaction," "Research in Context," "Factsheet," and "Science Response." The choice of a particular format is based on several factors. These include whether the issue originated in science or the public debate; whether there is scientific consensus or rapidly progressing insights; whether journalists are perceived to be in need of immediate, gradual or eventual orientation; and lastly, whether the issue can stand on its own or requires contextualization within larger scientific or societal developments. Finally, the editorial team considers the impact they want the broadcast to have, e.g., setting the journalistic agenda, debunking sensationalist claims, or providing comprehensive background information. In response to COVID-19, SMC Germany developed three new formats. These include "Virtual Press Briefings" for which editors invite high-level scientific experts and accredited journalists to discuss current scientific topics:

So that's why we've also ramped up these press briefings, because then we simply create a space where journalists can get their questions out, and they don't all have to call Prof. Drosten or anyone else individually. [...] *That was a point where we could really help the experts.* At the same time, journalists especially from smaller news media may fall behind [getting answers] from press offices, *but we were still able to offer them a platform to deliver their questions to the experts*. (Interview, editor, October 2020)

In addition, SMC Germany started the weekly "Corona Report," featuring data visualizations and explanations of statistical terms, in response to the need for basic statistical literacy among journalists during the COVID-19 pandemic (see for example Nguyen et al. 2021). The editors also introduced the "Annotated Publication List" in which the editors summarized and classified new research, including preprints, based on credibility and relevance.

#### **.. Expert statements**

SMC Germany's broadcasts rely on expert statements provided by scientists, many of whom are listed in the organization's expert database. In accordance with the expert factors of competence, prominence, eloquence, accessibility and reliability, and media experience, as described by Nölleke (2013, p. 275), SMC Germany editors prefer experts who actively engage in research within their field of expertise, have published in international journals, and have no conflicts of interest. This means that experts from academic research institutions are preferred, though exceptions can be made for "stories" on energy and mobility, where some experts are affiliated with private enterprises. After working with a particular expert, the editors also take into consideration their ability to communicate clearly.

Once suitable experts have been identified, SMC Germany editors reach out to them with an e-mail summarizing the key points and relevance of the issue. This e-mail often forms the basis for the later broadcast. Then, the editors formulate questions, asking the experts to evaluate scientific claims made in the public debate, to assess the methodology of scientific studies, and to consider the potential implications of new findings:

You have to make sure that your question is as broad as possible. We want to avoid statements that give a distorted view of what people in science think, or what science thinks. Because that [nuanced view] is what matters in the end. And that's not so easy, because we have a conflict of goals there. (Interview, editor, January 2020)

In general, it is important that more than one expert replies to SMC Germany's statement requests, and in the case of scientific publications, do so before the embargo falls:

The response rate of our experts is usually very high. And people [scientists] are resilient. They don't say, "Oh, my goodness, you again. Instead, they see the significance, they are convinced of it, and then they sit down and make these statements in their spare time, even though they are overworked. (Interview, head of IDM department, October 2020)

The editors use a content management system (CMS) to create the editorial product. All "stories" are received by accredited journalists as e-mail text. The e-mail begins with a note in bold indicating whether the content is "for immediate release" or "embargoed." Journalists are addressed as "Dear colleagues." The editors use the inverted pyramid style to convey the topic and its relevance. This is followed by the statements of the experts, which remain mostly unedited. Cooperating scientists are listed with their names, titles, and affiliations. The editors have different strategies for organizing the statements, sometimes by the order in which they were received, alphabetically by the experts, or by the main themes of the commentary.

#### **. Distribution**

From an organizational perspective, SMC Germany does not have set deadlines based on printing or broadcasting times. After "stories" are ready for broadcasting, the editors determine the optimal timing by considering journalists' schedules and the current public debate, and by anticipating potential influential events, e.g., political calendars.

SMC Germany relies on digital media such as e-mail, web conferencing, online publishing, and social media to reach its audience. For embargoed material, editors release their content 24 hours prior to the embargo lift to allow journalists sufficient time to incorporate SMC Germany content into their reporting. If additional expert statements arrive, editors include these in updated versions of the broadcasts. One day after the initial broadcast, SMC Germany makes all content, including video recordings and audio transcripts of press briefings, available on its website and publicly announces this via Twitter. As an alternative, the SMC Germany editors may choose to retweet an older "story" if a similar discussion is reemerging in the public debate.

Despite receiving requests for personalized content (Thurman, 2011), SMC Germany did not personalize its distribution to the interests of individual journalists or groups of journalists. The main reason given during the fieldwork was that it was technologically difficult to achieve. While this may have changed, the option still poses a challenge: on one hand, personalized science content could aid specialized science journalists to sort through topics more efficiently. On the other, it could hinder SMC Germany's efforts in placing science issues on the journalistic agenda, since not every registered journalist would see all content.

#### **. Interpretation and feedback**

A final step is the evaluation of SMC Germany's publications for quality, working practices, and impact. The latter is assessed through a semi-automated process that searches German news sources for references to SMC Germany or its content. The scan includes titles, body text, and expert statements, which may have been fully or partially copied into news articles without crediting SMC Germany. After each broadcast, SMC Germany creates a news clipping, similar to those used in public relations (Dozier & Repper, 1992) that shows which news media have picked up expert statements. These are shared with the collaborating experts. Editorial meetings are an important setting for editors to discuss the more opaque impact of SMC Germany broadcasts. For example, editors may discuss whether the formats and timing of broadcasts were appropriate, and whether they appear to have influenced the overall quality of scientific reporting on a particular topic:

[Editor 3] "*Spektrum* too."

[Editor-in-chief] "Was it right to do something about it, Editor 2?"

[Editor 2] "I had first only read that the Australians [SMC Australia] had done something about it and thought oops, that's quite an issue. Especially because of the amount of plastic particles."

[Editor-in-chief] "You find it [microplastic] more and more, and everywhere, so it's not really surprising. But we don't know what kind of impact it has. Now there is a study that says

<sup>[</sup>Editor 1] "The story on microplastics in baby bottles [Li et al. 2020] got very wide coverage, including *Deutschlandfunk*, *AFP* and *dpa*, many using our experts but some without." [Editor 2] "The *NTV* app as well."

particles can do something in the body but we don't know what exactly. But the amount is indeed surprising."

[Editor 2] "The image "baby with bottle" just sticks. It wouldn't be wrong to report on it again. It's worth it in individual cases. Here I think it is."

[Editor 1] "The experts also said that you don't automatically have to assume that it's mega harmful."

[Editor-in-chief] "*It would be interesting to see whether that message is also taken up or whether they've only taken over the scandal. We'll have to do some reading into that*." (Excerpt field notes, editorial meeting, 20 October 2020)

In addition, editors try to identify potential gaps in their access and monitoring routines: what relevant issues were missed? Why were they missed, and should there be action taken to ensure that this doesn't happen again? Finally, SMC Germany also receives feedback from its audience, either in response to "stories", or as the result of surveys sent out to ask journalists specific questions, e.g., which formats are most helpful for your work? In which ways do you use our content in your reporting?

### **Discussion**

In each of the news production stages, SMC Germany's editors create knowledge according to certain *"rules, routines and institutionalized procedures*" (Ekström 2002, p. 260, original italics) that share many similarities with science journalism in the digital age (Dunwoody, 2021). These result in science stories that are specially curated for further dissemination through journalism. The literature on epistemic journalism typically divides the knowledge involved in newsmaking into explicit knowledge related to specific expertise, and implicit knowledge about news value (Westlund & Ekström, 2020, pp. 81–82). At SMC Germany, the distinction between these two types is not always clear: editors apply knowledge of scientific topics and the inner workings of science on the one hand, and knowledge of current media debates and the structures of journalism on the other. Explicit knowledge is important because editors are expected to be well-versed in their areas of scientific expertise in order to identify important developments and false claims. In this way, SMC Germany's routines resemble "knowledge-based journalism" (Donsbach, 2014; Patterson, 2013). In addition, the organization makes explicit what usually remains tacit, for example by applying concrete social, journalistic, and scientific relevance criteria to its topic selection routines. These routines assist SMC Germany with constructing objectivity and congruence in its decisions. However, as Ettema and Glasser (1998) have pointed out, the rules by which journalists justify their epistemic practices are context dependent. For SMC Germany, communal sense-making processes, like editorial meetings, are important settings for negotiating the "contexts of justification" (Lyne, 1981, p. 148) that determine which science topics will be made into news as "public issues."

Although SMC Germany is technically independent from cycles of regular journalism (Schlesinger, 1978), its routines follow a clear sequence in which it attempts to marry the temporal contradictions of science and journalism. SMC Germany's routines facilitate "cooperation and collaboration" with actors from outside the newsroom (Westlund & Ekström, 2020, pp. 77–78), including scientific journals, media organizations, and scientists. In this assemblage, SMC Germany tries to identify "events" in the public debate on science, and in the process of scientific research, thereby giving science issues "a central organizing idea or storyline that provides meaning to an unfolding strip of events" (Gamson & Modigliani, 1987, p. 143). Scientific knowledge itself is, however, a representation of selected parts of reality, inherently incomplete, and subject to revision as new knowledge emerges (Bauer, 2017; Fleck, 1981, pp. 149–150). Since SMC Germany not only aims to anticipate but also create "events," its broadcasts are timed to match the routines of the journalistic audience in order to achieve maximum agenda-setting (or blocking) effect in the public debate. The criteria of "public issue" helps editors negotiate long-term developments in science with current knowledge needs in society. However, SMC Germany's routines rely in large part on relevance assignments constructed by actors within science, e.g. through press releases or embargoes. The expertise offered by collaborating scientists provides SMC Germany with "pre-established" facts (Ettema & Glasser, 1984, p. 10). These are afforded with high explanatory power in reference to science's own professional routines for knowledge production, and provide the knowledge produced by SMC Germany with epistemic authority. As such, SMC Germany broadcasts present a version of reality that is informed by an understanding of science as a cultural and institutional practice capable of producing reliable knowledge (Gieryn, 1999). Its focus on large journals in the medical and life sciences, similar to regular science newsrooms, could furthermore serve to exacerbate rather than reduce medialization tendencies.

Through its routines, selection criteria, editorial formats and situated practices, SMC Germany thus produces a specific kind of knowledge that combines substantive and procedural scientific and journalistic knowledge. It is with this knowledge that SMC Germany assigns or negates relevance, quality and timeliness to scientific topics and the expertise of scientists, communicates with its scientific partners and journalistic public, and ultimately contributes to the construction of science news .

### **Conclusion**

SMC Germany is considered an innovative organization (Buschow et al., 2022; Suhr et al., 2022) that fits the concept of "novel interfaces" (Guenther & Joubert, 2021) that have emerged in response to horizontal flows in science communication (Franzen, 2019). Its routines resemble those of traditional newsrooms with a division into "beats," an editorial hierarchy, and news production practices. SMC Germany contributes to the construction of science news by limiting its scope of scientific topics and sources, assigning science issues with contextualized scientific, social, and journalistic relevance criteria ("public issue"), enriching its content with authoritative expertise, and timing its broadcasts with the intent to promote, alter or prevent journalistic coverage of scientific issues. Through these routines, SMC Germany produces knowledge that negotiates journalistic and scientific norms. The organization is able to take on "outsourced" practices of science journalism due to a combination of epistemic, practical, and organizational advantages. Its editors possess both implicit and explicit knowledge of scientific fields and the workings of science and journalism, can make use of automated processes, and have autonomy in editorial decisions without strong financial and temporal pressures. However, SMC Germany reflects some structures of regular science journalism, such as its narrow focus on medical, environmental, and technological research and its reliance on the embargo system, which may perpetuate tendencies of medialized and strategic science communication. Further research, such as content analyses comparing the output of newsrooms registered with SMC Germany to that of those not registered, is needed to gain insight into this.

### **Translations**

All of the excerpts have been translated from the original German into English language with the help of the DeepL software.

## **Funding**

The ethnographic fieldwork was carried out as part of the research project "MeWiKo - Medien und wissenschaftliche Kommunikation" financed by the German Federal Ministry of Education and Research (BMBF) under the funding code 01PU17O18D.

### **Acknowledgements**

I would like to thank the editors of SMC Germany for their openness during the ethnographic fieldwork, Louisa Pröschel for her assistance with data analysis, and Alice Fleerackers for her constructive feedback after reviewing an earlier version of this chapter.

### **Data transparency**

The ethnographic material is available for conditional reuse for scholarly purposes (Broer, 2021, 2022).

### **References**


Irene Broer



Nelkin, D. (1987). *Selling science: How the press covers science and technology*. W.H.Freeman & Co Ltd.

Neuberger, C., Bartsch, A., Reinemann, C., Fröhlich, R., Hanitzsch, T., & Schindler, J. (2019). Der digitale Wandel der Wissensordnung. Theorierahmen für die Analyse von Wahrheit, Wissen und Rationalität in der öffentlichen Kommunikation. *Medien & Kommunikationswissenschaft, 67*(2), 167-186. https://doi.org/10.5771/1615-634X-2019-2-167

Nguyen, A., Zhao, X., Lawson, B., & Jackson, D. (2021, January 26). *Reporting from a statistical chaos: Journalistic lessons from the first year of covid-19 data and science in the news* [Monograph]. Bournemouth University, Royal Statistical Society and Association of British Science Writers. http://eprints.bournemouth.ac.uk/35109/

Nölleke, D. (2013). *Experten im Journalismus. Systemtheoretischer Entwurf und empirische Bestandsaufnahme*. Nomos. https://doi.org/10.5771/9783845245317

Nowotny, H., Scott, P., & Gibbons, M. (2001). *Re-thinking science: Knowledge and the public in an age of uncertainty* (Repr). Polity Press.

Patterson, T.E. (2013). *Informing the news: The need for knowledge-based journalism*. Vintage Books, A Division of Random House LLC.

Peters, H.P. (2021). Scientists as public experts: Expectations and responsibilities. In *Routledge handbook of public communication of science and technology* (3rd ed.). Routledge.

Raupp, J. (2017). Strategische Wissenschaftskommunikation. In H. Bonfadelli, B. Fähnrich, C. Lüthje, J. Milde, M. Rhomberg, & M.S. Schäfer (Eds.), *Forschungsfeld Wissenschaftskommunikation* (pp. 143-163). Springer Fachmedien. https://doi.org/10.1007/978-3-658- 12898-2\_8

Ritter, W.E. (1926). Science service and E.W. Scripps' philosophy of life. *The Science News-Letter, 10*(298), 201. https://doi.org/10.2307/3902922

Rödder, S. (2020). Organisation matters: towards an organisational sociology of science communication. *Journal of Communication Management, 24*(3), 169-188. https://doi.org/10.1108/JCOM-06-2019-0093

Ruß-Mohl, S. (1993). Konjunkturen und Zyklizität in der Politik: Themenkarrieren, Medienaufmerksamkeits-Zyklen und "lange Wellen". In A. Héritier (Ed.), *Policy-Analyse: Kritik und Neuorientierung* (pp. 356-368). VS Verlag für Sozialwissenschaften. https://doi.org/10.1007/978-3-663-01473-7\_16

Ryfe, D. (2016). News routines, role performance, and change in journalism. In *Journalistic role performance* (pp. 127-139). Routledge.

Ryfe, D. (2018). A practice approach to the study of news production. *Journalism, 19*(2), 217- 233. https://doi.org/10.1093/oxfordhb/9780190497620.013.5

Schlesinger, P. (1978). *Putting "reality" together. BBC news.* Constable and Company.

Schudson, M. (1989). The sociology of news production. *Media, Culture & Society, 11*(3), 263- 282. https://doi.org/10.1177/016344389011003002

Schultz, I. (2007). The journalistic gut feeling. *Journalism Practice, 1*(2), 190-207. https://doi.org/10.1080/17512780701275507

Science Media Center Germany. (2021). *Gesellschaftsvertrag*. https://www.sciencemediacenter.de/das-smc/wer-uns-kritisch-begleitet/

Science Media Centre U.K. (2012). *SMC charter*. https://www.sciencemediacentre.org/wp-content/uploads/2012/07/SMC-Charter-003.pdf


### Arno Simons and Alexander Schniedermann **3. Preprints in the German news media before and during the COVID-19 pandemic. A comparative mixed-method analysis**

**Abstract:** Mainstream media widely references scientific publications for claims of factuality and authority. But how did science journalism deal with the sudden surge in preprint publications that provided rapid but often uncertain knowledge during the COVID-19 pandemic? While several studies have investigated various aspects of preprint-based science communication, only a few have focused on the public discourse in Germany, albeit with substantial challenges and controversies. In this mixed-method study, we identified the usage of preprints for 1,006 in about 390,000 German news stories, qualitatively analyzed the contexts of these preprints, and developed codes that reflect the epistemic sentiments. We further compared the code compositions of news stories that cover the pandemic with those about other topics. We found that the amount of news stories which used preprints increased with the pandemic. Frequent framings of preprints include accessibility, timeliness, and uncertainty, where the latter was more prominent in corona-related than coronaunrelated news stories. Beside using preprints as sources for claims, some news stories referred to them as a publication genre to turn scientific publishing into a story itself. Based on our findings we argue that journalists have to be transparent about their usage of preprints as well as reflect the benefits and drawbacks of using them.

**Keywords:** health communication, preprints, German press, scientific uncertainty, mixed-methods

### **Introduction and research question**

Over the last decade, journalism in general and science journalism in particular have been changing in response to digitalization as well as to new demands for immediacy and transparency (Dunwoody, 2021; Allan, 2009). While journalism has always been profoundly about "getting it right," truth-telling strategies and standards are currently being re-negotiated (Craig, 2016; Le Masurier, 2015; Karlsson, 2011). The outbreak of the COVID-19 pandemic has only propelled this development, and it has put the truth-telling role of journalism to a new test (Dunwoody, 2020). On the one hand, the pandemic has revealed the difficulties involved in formulating and legitimating wide-ranging policy responses to a problem whose nature is inherently scientific while our scientific knowledge of it is highly uncertain (Bicchieri et al., 2021; Parviainen et al., 2021; Kreps & Kriner, 2020). On the other hand, the pandemic has spurred an unprecedented growth and acceleration of scientific research and publishing in the biomedical field and beyond (Wang & Tian, 2021; Horbach, 2020; Torres-Salinas, 2020), resulting in a number of highly appreciated discoveries and inventions such as in the content of new mRNA-vaccines (Dolgin, 2021).

In this context, an interesting phenomenon related to both the general transformation of journalism and the specific development of pandemic science and its media coverage is the dissemination and uptake of academic preprints. Preprints can be defined in several ways depending on their stage in the conventional publication process (Till, 2001). We use the term to denote academic papers that are published online ("e-preprints"), on dedicated preprint servers or other openly accessible outlets, at a time when they have not (yet) been peerreviewed in a process typically organized by academic journals.

For journalists, preprints are both tempting and risky. On the "pro-side," preprints provide fast and free access to the latest scientific findings, which became especially relevant during the pandemic. Of the 125,000+ COVID-19 related scientific papers released within 10 months of the first confirmed case more than 30,000 were hosted by open-access preprint servers before they were published by peer-reviewed journals (Fraser et. al, 2021; cf. Colavizza et al., 2021). The dissemination of corona-related preprints not only accelerated the scholarly discussion of scientific results but it also allowed for faster journalism and a more timely information of pandemic policymaking (Fleerackers et al., 2021; Fraser et al., 2021; Horbach, 2020). On the "con-side," preprints lack academic peer review, and so their findings have to be taken with extra care, especially when interpreted by non-scientists, including journalists (Fleerackers et al., 2021; Chiarelli et al., 2019). In this light, preprints may "change the rules of the expertise game" (Heimstädt, 2020) with an effect also on the way science is reported in the media.

How do journalists deal with preprints, especially when they become more important within science as well as for informing policy? In particular, how do journalists frame preprints with regard to their advantages and disadvantages as discussed by scientists, policymakers, and journalists? Here we present results from a systematic qualitative and quantitative case study on the coverage and framing of preprints in 1,006 German news stories identified in 390,942 stories issued by seven major German news outlets between 2018 and 2021. We

compare the use and reporting of preprints before and after the beginning of the pandemic and in news context both related and unrelated to COVID-19.

#### **Preprints in science and in the media**

While the surge in and public visibility of preprints during corona is unprecedented, preprints have been around in science for a very long time (Moore, 1965). The first large-scale exploration of preprinting occurred in the 1960s during a six-year experiment with so-called Information Exchange Groups (IEGs), the members of which exchanged hard-copies of preprinted manuscripts via mail, financed by the US National Institutes of Health (NIH). The organizers and participants of the IEGs, as well as of similar initiatives in the physical sciences around the same time, were generally happy with these services, but strong opposition came from academic publishers and leading journals, such as Science and Nature, who feared that their business model was in danger (Cobb, 2017; Till, 2001). Only much later, with the development of new editing software, the advent of the World Wide Web (WWW), and the creation of the preprint server arXiv in 1991 supported by the National Science Foundation (NSF), did preprinting really take off as an alternative publishing practice in physics and increasingly in other disciplines.

Today, preprinting is not only "a device for quick scientist-to-scientist communication" (Moore, 1965, p. 127), but more generally a tool for accelerated science communication to both internal and external audiences, including journalists.1 The two key advantages of preprinting are easy and fast dissemination of research findings and better accessibility (Chiarelli et al., 2019). This combination of timeliness and accessibility makes preprints attractive also to journalists looking for new and accessible information and sources.

The biggest caveat with preprinting is the lack of formal peer review. The peer-review process organized by academic journals is meant to ensure quality standards, improve performance, and provide credibility of a manuscript. Peer review thus functions as a filter for quality and relevance, especially for external audiences like journalists and policymakers, who are not generally capable of assessing the value of a scientific manuscript. Since preprints have not gone

 **1** The timeliness aspect of preprinting also plays a role in the establishment of priority claims. In several cases, scientists have used preprints as a tool to outpace competitors during otherwise lengthy publication processes, sometimes linked to the sharing of preprints with the media as a social amplifier (Weingart, 1998; Lewenstein, 1995; Nelkin, 1995).

through this filter, they are typically associated with greater scientific uncertainties than peer-reviewed publications in academic journals (Chiarelli et al., 2019).

Science journalists are increasingly aware of these issues. Since 2019, the stylebook of the Associated Press (2019), a leading reference for journalists around the world, emphasizes that any "research that has not been peerreviewed, including articles posted on preprint servers, should be reported with extreme care," and links this to the remark that science reporting "comes [with] unique responsibility. A misleading or incorrect story could lead someone to make unwise, harmful choices."2

To sum up, preprints are both tempting and risky for journalists: tempting because of their timeliness and accessibility, risky because of the scientific uncertainties due to the lack of peer review. According to Heimstädt (2020),

it is up to journalists and policymakers to familiarise themselves with the most important preprint servers and their specific moderation techniques (e.g. sanity checks of uploaded preprints by a small editorial team). Only when understanding the governance of such new and more open scientific practices will they be able to leverage the benefits of fast science while avoiding the threat of disinformation.

### **Sources, factuality, and framing**

The use of preprints as sources in news stories is (a new) part of a set of journalistic practices called "truth-telling," aimed at generating a sense of "factuality" (Pan & Kosicki, 1993; van Dijk, 1988; Tuchman, 1980). According to Pan and Kosicki (1993, p. 61), news stories are often characterized by a "hypothesistesting (or research finding) aspect," which is especially true when they report about research-related topics such as COVID-19. But often there is "no clear distinction between factuality and persuasion. The rhetorical claim of news being factual and impartial helps establish the epistemological status of news as a source of factual information and the authority of news as a mirror of reality" (p. 62).

 **2** That preprints "should not be reported in news media as established information" has also been demanded by leading preprint servers during the pandemic. Such warnings have been posted, for example, on medRxiv's website since their launch in June 2019 (https://web. archive.org/web/20190630063933/https://www.medrxiv.org/) and on bioRxiv's epidemiology page since February 2020 (https://web.archive.org/web/20200202105940/https://www. biorxiv.org/collection/epidemiology).

As van Dijk (1988, p. 84) highlights in his book *News as Discourse*, "If propositions are to be accepted as true or plausible, there must be special means to enhance their appearance of truth and plausibility. News discourse has a number of standard strategies to promote the persuasive process for assertions." Such strategies of persuasion—which resemble practices in science publication (Latour, 1987; Fleck, 1980)—include the use of authoritative sources, direct quotations, and discursive markers indicating precision and exactness, such as numbers or particular adjectives (Pan & Kosicki, 1993; van Dijk, 1988; Tuchman, 1980).

The construction of "factuality" in media reporting (as well as in science) can be analyzed in terms of framing (Pan & Kosicki, 1993; Entman, 1993; van Dijk, 1988; Latour, 1987). Pan and Kosicki (1993, p. 59) define a news story's "frame" as "an idea that connects different semantic elements of a story (e.g., descriptions of an action or an actor, quotes of sources, and background information) into a coherent whole." Whereas the frame intended by the journalist never perfectly matches the one comprehended by the reader (van Dijk, 1988), on the writing side, the choice of conventionally understandable structural and lexical "framing devices" opens "a cognitive 'window' through which a news story is 'seen'" (Pan & Kosicki, 1993, p. 59).

While truth-telling likely "is the most essential component of journalism… the means by which this can be accomplished… change radically in the digital environment" (Karlsson, 2011, p. 279). One such change, according to Karlsson, is the trend toward immediacy in an ever accelerating news cycle. Speed and timeliness generally provide a comparative advantage in journalism (Craig, 2016). But with the increased acceleration of online journalism, timeliness comes with a caveat. The trend toward immediacy increases the likelihood of provisory, incomplete or dubious news reporting (Craig, 2016; Le Masurier, 2015; Karlsson, 2011). This speed-accuracy tradeoff is one of the factors that has led to a restructuring of journalistic authority. To maintain journalism as an authoritative source of information, many authors have called for transparency as a new publicly communicated standard for establishing accountability and legitimacy (Karlsson, 2011; Allen, 2008).

These developments have been reflected in recent research on media framing of science in general and of scientific publications in particular. Studies find, for example, that news outlets increasingly point their readers to academic sources via hyperlinks to demonstrate credibility and transparency (Stroobant & Raeymaeckers, 2019; Karlsson & Sjøvaag, 2018). Research on the framing of scientific uncertainties reveals a mixed picture. In a study of 149 news stories in nine major US and Canadian online news outlets, Matthias et al. (2020, p. 1) find

that academic sources are mostly framed as "certain," only sometimes as "controversial" and least often "uncertain." Dumas-Mallet et al. (2018) find that most of 426 news stories covering 40 initial biomedical studies frame these studies as "initial" but only 21% mention that they should be "confirmed by replication." A systematic, quantitative content analysis by Guenther et al. (2019) of 128 science stories published in seven major German media outlets reveals that while scientific findings are predominantly depicted as "certain," uncertainty framings are more common within specifically dedicated science sections or in stories with natural scientific or medical scientific content. We found only one study on the media framing of preprints.

Fleerackers et al. (2021) studied the mentions of 100 preprints in 457 news stories and found that about half of the references to preprints in their sample contained one or more of four "uncertainty framing devices": formulations that the cited study was 1) a "preprint"; 2) "unreviewed"; 3) "preliminary"; and/or 4) "in need of verification."

Our analysis complements current research by combining traditional science media studies and more recent approaches that utilize large-scale quantitative data. In the latter case, researchers select scientific publications and search for their media coverage, often by utilizing large-scale databases such as Altmetric (Fleerackers et al., 2021; Matthias et al., 2020). Although this data-driven approach provides access to a huge number of publications and media items, it is mostly based on standardized links via URLs/DOIs or via bibliographic information such as author names, publication year or journal title which can be matched with other bibliographic sources.3 In addition, it has been found that such databases are often biased towards recency, English language, and particular disciplines. In contrast, more traditional science media studies start their analysis by sampling news outlets (Guenther et al., 2019; Riles et al., 2015) or journalistic databases (Dumas-Mallet et al., 2018). Such approaches can identify situations in which scientific publications cover other formats, for example expert interviews. But in contrast to larger datasets, they are usually limited in their timeframe or selection of news outlets, which makes them less suited to cover longer and broader events like the pandemic in a longitudinal manner.

 **3** For some of their sources (news, policy, and patents), Altmetric additionally uses text mining: https://help.altmetric.com/support/solutions/articles/6000240263-text-mining

### **Data and coding**

Our data pipeline consisted of four steps. First, we selected the time span and the sources. We picked the 48 months long period between January 2018 and December 2021 inclusive, to be able to compare the coverage of preprints before and after the outbreak of COVID-19 in December 2019/ January 2020. The selection of sources was based on three criteria: their type, their circulation, and their accessibility. We wanted to include both major daily and weekly newspapers as well as major online news providers. Accessibility was an issue because we had no institutional access to any of the targeted sources, most of which contain at least some paid content. In several cases, we could leverage trial subscriptions to get past a paywall, but in other cases there were simply no affordable pricing options.


**Table 1:** Retrieval strategy and coverage per source (sorted by number of retrieved stories).

Second, we identified and downloaded retrievable news stories of interest. By retrievable we mean stories for which we could get hold of their metadata, including most importantly their URL but also the title and the date of publication. Generally, we aimed at retrieving the total population of stories published by each source in the study period. However, this was only possible for three sources: SPIEGEL, SPON, ZEIT. For all other sources, in which the total population was unknown to us, we developed strategies for searching and retrieving all stories mentioning preprints and other types of scientific publications (subpop-1) and, given that preprints have increased dramatically during the pandemic, all stories related to COVID-19 (subpop-2). Depending on a source's specific search capability and limitations, we defined multiple and sometimes complex queries to retrieve these two subpopulations. For all but one source (BILD), we are highly confident to have retrieved these subpopulations. Since BILD limits its search results to 1,000 articles, we used a large number of connected keywords including "preprint AND corona," "preprints AND physics," "corona AND masks," "corona AND vaccines," etc. to narrow down the search space, hoping to retrieve as many news stories as possible. In the end we are still only medium confident to have achieved our goal of retrieving the two subpopulations.

**Figure 1:** Number of news stories per source and year (N = 390,942, all sources).

We downloaded all retrieved stories and compiled a dataset containing the metadata, the html data, and the extracted plain text data. All this was done with scripts that we coded in Python, using standard libraries as well as external libraries like *Requests* and *BeautifulSoup* for html parsing, *Pandas* for data management and analysis, and *Matplotlib* for visualization. Our final dataset contains 390,942 news stories from seven major German news outlets. Table 1 lists the overall retrieval strategies and coverage per source. Figure 1 plots the number of retrieved stories per source and year.

Third, we labeled all stories according to two categories of interest. First, we call stories "p-stories" if they relate to preprints in one of the following ways: either a) their plain text contains any of the terms: "preprint," the German syn-

onyms "Vorveröffentlichung," "Vorab-Veröffentlichung" or the name of one of over 50 preprint servers4 ; or b) their html includes at least one hyperlink to a URL that contains the substring "preprint" or, again, the name of one of over 50 preprint servers. Second, we made a distinction between "corona stories" and "non-corona stories." All stories whose plain text contains any of the terms "corona," "COVID-19," "2019-nCov" or "cov-2" went into the former category, or all stories went into the latter.

**Figure 2:** Number of p-stories (N = 1,006, all sources).

Figure 2 shows the distribution of 1,006 identified p-stories over time. The four bars represent absolute numbers of p-stories per given year of our four-year study period. The colors indicate how many of the p-stories were related (orange) or unrelated (blue) to corona reporting. As expected, we find that the number of p-stories rose steeply with the start of the pandemic. The increase from 2018 (58) and 2019 (58) to 2020 (449) and 2021 (451) amounts to a multiplication close to factor 8. As indicated by the coloring, the absolute majority of pstories in 2020 (91%) and 2021 (92%) were focused on corona.

 **4** To be more precise, all of our dictionaries contained regular expressions. For example, to cover the names of over 50 preprint servers, we used shortcut expressions such as r"[rin]xiv(?!m)" to include servers that contain "xiv" while excluding false positive matches such as "NXIVM," an American cult, which engaged in sex trafficking, forced labor, and racketeering, and which was covered in the German press during our study period.

Finally, we qualitatively analyzed all 1,006 p-stories on how they framed preprints. While our analysis was guided by the methodology and results of previous studies, especially Fleerackers et al. (2021), we started with a few rounds of open coding, allowing us to "get a feel" for the ways in which preprints are framed in our sample. We then gradually moved from open codes to more defined and settled codes by iteratively comparing, refining, splitting, and merging our codes until we reached a good level of intercoder agreement and overall saturation. Eventually, we defined 12 codes of interest relating to the following five concepts: 1) framing, 2) naming, 3) referencing, 4) genre, and 5) focussing. In the following sections we present these concepts and associated codes both qualitatively and quantitatively as findings, since that is what they really are, the results of an iterative, theory generating coding process (cf. Glaser & Strauss, 1967).

#### **Framing and naming**

Given the controversiality of preprints among researchers and journalists, we analyzed to what extent the discussed pros and cons of preprints can also be found as framing devices in our sample set of p-stories. On the con-side we found uncertainty framing devices quite similar to those reported by Fleerackers et al. (2021). Several p-stories in our sample frame preprints as "unreviewed," "preliminary," "premature," or "in need of verification," sometimes linked to warnings that preprints "may lead to the circulation of unscientific claims" or "fuel bad policy decisions." We coded such stories as uncertainty. On the proside we found p-stories associating preprints with notions of timeliness or access, which we also used as codes.5 We especially noticed emphases on the recency of preprint studies, expressions like "a preprint study published on Wednesday" (2021, SUED) or "the results have just been published on a preprint server" (2021, SPON). In one case, a brand new preprint was added to a story via an update to demonstrate a concern for immediacy: "Update: In the story it says the results of the recovery-study… are not yet available. This has changed since the publication of our story. In the meantime, there is a preprint: MedRxiv: Horby et al., 2020" (2020, ZON).

 **5** To be able to analyze "pro-framing" in comparison to "con-framing," we will sometimes use a combined code timeliness\_or\_access for the former. The latter then is equal to the uncertainty code.

When highlighting access, p-stories contain formulations like "The complete paper is available on the preprint server PsyarXiv" (2021, ZON). The following example shows a strong double framing of both timeliness and access: "In their publication, which has been freely accessible to everyone on the preprint server 'arXiv' since Thursday, the researchers provide the standard deviation of 4.8 sigma" (2018, FAZ).


**Table 2a:** Corona p-stories. Annual distribution of codes.

In terms of numbers, we see striking differences between corona p-stories and non-corona p-stories. Tables 2a and 2b, which show the distribution of codes over time for these two categories, as well as Tables 3a and 3b, which show the cross-tabulation of codes, reveal that uncertainty framings occur more often in corona p-stories than in non-corona p-stories. Comparing only the total columns in 2a and 2b, we see that the percentages of timeliness (19% in 2a vs. 20% in 2b) and access (4% in 2a vs. 5% in 2b) do not change much in relation to coronacoverage, while uncertainty framings are more than twice as frequent in corona articles (38% in 2a vs. 14% in 2b). When paying attention to time, we find that uncertainty framings have doubled in non-corona p-stories from 2020 (10%) to 2021 (22%), which we interpret as a result of a more intense discussion of the scientific uncertainties surrounding preprints related to corona preprints.

Tables 3a and 3b reveal that of all non-corona p-stories that use either of the three framings (uncertainty, timeliness, or access) (any\_framing) 76% include pro-framings (timeliness\_or\_access) and only 44% include uncertainty framings, whereas this relationship is reversed in corona p-stories (44% timeliness\_or\_access vs. 81% uncertainty).


**Table 2b:** Non-corona p-stories. Annual distribution of codes.

When analyzing how preprints were framed in terms of uncertainty, timeliness, and access, we felt a need to differentiate between framing and naming. While we interpret the word "preprint" or its direct German translations as names that can be used for any particular preprint or for the preprint genre in general, we did not automatically interpret such names as framing devices. In particular, we did not follow the suggestion by Fleerackers and colleagues to interpret the presence of the word "preprint" as an uncertainty device: unlike "unreviewed," "preliminary," etc., the term "preprint" does not directly point to uncertainty or any aspect of it. Thus a link between the term "preprint" and uncertainty can only be evoked in the minds of readers already associating preprints with uncer-

tainty. However, by the same token readers may also associate preprints with timeliness or access or any other attribute. In other words, associating "preprint" with uncertainty requires not only a knowledge of what preprints are, but also of the pros-and-cons discussions related to the preprint genre. What is more, if any of the three framings under investigation could be said to be included in the name "preprint," it arguably is timeliness, because the prefix "pre" means something like "before" or "prior to."

The German language context adds even more complexity to the matter. When explicitly naming preprints, German journalists mostly use the English expression "Preprint," which we coded as name\_en. But sometimes they use a direct German translation. In such cases, which we coded name\_de, "pre" is translated as either "vor" or "vorab" while "print" is typically translated as "Veröffentlichung," "Publikation," "Studie" or "Druck." Like "Preprint," none of these translated names invoke aspects of uncertainty directly, which is why we did not treat them as uncertainty framing devices. However, in contrast to the rather general prefix "vor," the prefix "vorab" has a more distinct meaning, which can be translated as "in advance." Ironically, the key example sentence for the use of "vorab" provided by the *Duden*⸺Germany's authoritative dictionary of the Standard High German language⸺reads "Die Presse wurde vorab informiert" ("The press has been informed in advance"). We therefore decided to interpret the use of "vorab" (but not "vor) in German names for preprint as a timeliness framing device and coded such instances accordingly. We hesitated to also code "Vorveröffentlichung," "vorveröffentlicht" or "Preprint" as timeliness, because, as stated above, we believe that the meaning of the prefix "pre/vor" is less straightforward.

We assigned a combined code any\_name to all p-stories that use either an English or a German name or both. P-stories that don't include any proper name for preprints were coded no\_name. Note that such stories can still include descriptions and framings of preprints or of some of their aspects, e.g. in formulations like "the un-reviewed publication" or "a study, which has been released in advance via an online platform."

Naming practices have evolved over time. Naming is twice as common in corona p-stories compared to non-corona p-stories (Tables 2a vs. 2b: name\_en: 44% vs. 24%, name\_de: 8% vs. 4%, any\_name: 48% vs. 27%). The proportion of English names to German names fluctuates a bit over the years and across categories, but it has always been in favor of the English "Preprint," and it seems to have increased especially since 2021, possibly indicating that "Preprint" is about to become the dominant name also in the German language context.


**Table 3a:** Cross-tabulation of codes assigned to corona p-stories. Relative portions of code cooccurrences against absolute codes (diagonal). Read horizontal from row to column.

Our qualitative analysis of naming practices similarly points to the negotiation of how to best name and refer to preprints. For example, several p-stories contain expressions like "a so-called preprint," as if to educate the readership that preprints exist in the first place and that it can be legitimate to use them as sources in news stories. One context explains preprint naming by referring to a famous German corona-news podcast: "They are also known by the names 'working paper' or 'preprint'... this formulation has been popularized by the Drosten-Podcast" (2021, SUED). We interpret such educational interventions also as indications for a broader struggle within journalism of how to deal with preprints as sources, given their timeliness and accessibility but also their uncertainty.


**Table 3b:** Cross-tabulation of codes assigned to non-corona p-stories. Relative portions of code co-occurrences against absolute codes (diagonal). Read horizontally from row to column.

#### **. Preprints as traceable sources**

In the absolute majority of p-stories, preprints were used as sources to state and/or support claims or arguments. For each p-story we coded how many unique preprints it refers to, resulting in a total of 1,244 unique references distributed across p-stories as shown in Figure 3.6 The number of references per pstory is not correlated with corona reporting and remains similar over time. Overall, close to 80% of p-stories contain exactly one reference, between 10% and 20% of p-stories contain two references, and very few p-stories contain

 **6** Unique reference means that if a preprint was mentioned more than once in a story it was counted only once. We did not match references across p-stories, and thus cannot quantify the number of unique preprints cited in our sample. In Figure 3 we make use of lines merely to improve the readability of the plot, not to suggest that our numbers were continuous between the years.

three, four or more references. Most preprint references are traceable to their original source. We coded the traceability of preprints by checking if one of the following four minimal combinations of metadata was present in the text: 1) a DOI; 2) a hyperlink to the source; 3) a standardized scientific reference; or 4) authors' name(s) plus publication date plus either a complete title or at least a fair description of the content of the study. P-stories that contain at least one reference to a concrete preprint (min\_1\_ref) almost always enable the tracing of at least one of these preprints (min\_1\_trace), regardless of whether these pstories relate to corona or not (97% in Table 3a vs. 98% in Table 3b).

References to preprints can take different forms. First of all, such references can be explicit: "According to a preprint study from India, the transmissibility is 50% more than in the British variant" (2021, BILD). In rare cases, found especially in ZON, references to preprints even take the form of standardized citation styles, e.g., "Large studies from Israel (The Lancet: Haas et al., 2021), England and Scotland (Lancet Preprint: Vasileio et al., 2021) show that the vaccines prevent almost all symptomatic cases and even most infections" (2021, ZON).

**Figure 3:** Number of references to preprints per p-story over time and across categories.

But in the majority of cases preprints are more implicitly referenced, without being named. Less than half of the corona p-stories that reference at least one preprint and less than a quarter of the corona p-stories that reference at least one preprint also name preprints. This can be read off the cross-tabulation of min\_1\_ref and min\_1\_ref in Tables 3a (45%) and 3b (23%).

In cases where preprints are cited but not named, preprints are often simply called "studies," as in "Around 15 percent of all Twitter users are bots, or nearly 50 million! This is the result of a study" (2018, BILD). In other cases preprints are added via hyperlinks but the anchor text does not mention that the link goes

to a preprint, as in "the universe expands accelerated" (2019, FAZNET). Then there are cases where preprints are not named but some of their typical aspects are described. For example, the context "A not independently reviewed analysis from… Guangzhou estimates a value around 19 percent" (2020, SPON), which contains a hyperlink to a preprint, does not call its source a preprint but highlights that the source has not undergone peer review, invoking an uncertainty framing. Sometimes, p-stories also hyperlink to preprint versions of already published studies. We interpret such cases as indirect indications of the importance of public accessibility of sources for journalists.

#### **Preprints as a genre in science communication**

Preprints are not only used as (traceable) sources, they are also discussed as a distinct category, or genre, of the academic literature. In a broad sense, the preprint genre becomes invoked when a referenced study is called a preprint (any\_name). But our code for genre (genre) is more restricted in that it applies only when the genre is discussed more directly. A minimum case was the formulation "a so-called preprint," where the term "so-called" emphasizes that the cited preprint is but a "token" of a more general "type," i.e., the preprint genre. But more generally, we applied the code to p-stories that discuss general aspects of preprints, such as that their lack of peer review must be taken with caution or that preprints are becoming more frequent in science publishing. Illustratively, p-stories have covered the scientific publication system in general, featuring titles such as "Researchers have to publish constantly - this harms science" (2018, SUED).

Genre talk often seems to be linked to educational interventions. Already simple expressions like "a so-called preprint" inform the readership about a new name and concept. Other contexts further explicate that preprints are "a kind of discussion material" (2020, SPON), that they are "currently common in science" (2020, SUED) or that they are "publications, which have not yet taken the high hurdle of being positively reviewed in a disciplinary journal" (2021, ZON). More extensive examples are given below.

In terms of overall numbers, we again see differences between corona and non-corona p-stories. The genre code is more present in the former group than in the latter, and in both groups occurrences decrease over time. A total of 22% of corona p-stories include genre talk in 2020, compared to 13% in 2021. Around 10% of non-corona p-stories addressed the preprint genre in 2018 and 2019, but only around 5% did in the following two years.

In p-stories that describe preprints as a genre, the balance between "pro-" and "con-framings" changed with the pandemic. In non-corona p-stories, genre coincides roughly equally with both uncertainty (53%) and timeliness\_or\_access (59%), whereas in corona stories, genre coincides much more often with uncertainty (80%) than with timeliness\_or\_access (44%). This supports our general finding that uncertainty framings occur more often in corona p-stories than in non-corona p-stories.

In some p-stories the preprint genre is discussed as a distinct topic of its own, unrelated to concrete examples or a referenced source. Cases of genre without any reference to a preprint (min\_1\_ref = false) are found more often in non-corona p-stories (100% - 35% = 65% in Table 3b) than in corona ones (100% - 72% = 28% in Table 3a). This result is mainly due to a number of p-stories in 2018 and 2019 discussing overall changes in pre-pandemic science publishing practices, especially in physics and biomedical sciences. Ten of these stories amounting to almost 12% of all p-stories in these two years—debate the pros and cons of preprints without citing a single concrete preprint. One story speculates that preprints could be the "ideal future" (FAZNET, 2018) of science publication, stressing the pioneering role of physics and biosciences in this regard and quoting a Nobel Laureate as envisioning a world in which all publications are free and post-publication peer-reviewed. Another story speaks of preprint publishing as a form of "basic democracy," where peers collectively and openly decide upon the quality of a study, but it also notes that through open preprint servers "journalists, policy makers, and lobbyists alike get hold of premature and unreviewed publications, and possibly spread or mobilize half-cooked, false or interest-driven results" (SUED, 2018). This discussion indicates that, already before the pandemic, the emerging role of preprints as a genre in science communication had become a newsworthy topic in the German media.

In later corona p-stories, we observe a profound change of tone in genre debates. Due to the overall explosion of preprints and their new importance for informing policymaking, journalists discussed preprint publishing much more critically. Whereas the earlier genre debates positively highlight free and fast access and collective post-publication review, pandemic genre debates stress the downsides of preprints when accessed by non-scientists and especially when political decisions are based on their results. One p-story highlights, for example, that "especially in relation to the new coronavirus this intensive exchange has a downside" (2020, SUED). Another one explicitly juxtaposes the different epistemic risks associated with physics preprints in comparison to medical ones, which public health decisions are based on:

As long as preprint servers were relevant only to academia, this did not mean a problem: with some delay, the self-control of science eventually worked. Meanwhile the instrument has begun to suffer from its own popularity… In the case of a sloppy study from physics, media sensationalism rarely has negative consequences for the general public… but this can be different in the life sciences. This became clear several times in the corona crisis, for example when a study overestimated the number of unreported infections in the USA or when politicians jumped to conclusions from poorly done experiments (2021, ZEIT).

#### **Preprints in focus**

In some p-stories the focus is so much on a particular preprint or on preprints in general that they are, in fact, stories about preprints, rather than stories merely mentioning or using preprints. We coded such stories as p\_in\_focus.

P-in-focus-stories typically contain some sort of reference to preprints in their headlines. "Studies present first results on the danger of the British variant B.1.1.7" (2020, SPON) and "Study shows why all hipsters look alike" (2019, ZON) are two example headlines of p-stories focussing on particular preprints. Example headlines of p-in-focus-stories targeting the genre as a whole are "Scientific Results Become Freely Available" (2019, SUED) or "Studies on the Coronavirus: Stress Test for Science" (2020, SUED).

A special case in this regard is the German debate about a particular preprint by Christian Drosten and colleagues, which not only employed various framings of preprints in general, but also led to a broader discussion about science journalism during the pandemic. The Drosten preprint argued that children were as infectious as adults and cautioned "against an unlimited reopening of schools and kindergartens" (Jones et al., 2020, p. 6), thereby providing substantial fuel for political conflict. One month after its publication, BILD wrote that the Drosten study was "grossly false," allegedly based on expert opinions uttered on various platforms. Although this led to a criticism of the journalistic methods used by BILD, it nevertheless triggered a public controversy about the credibility of Christian Drosten. In September 2020 the German press council reprimanded the BILD story, deciding that the wording "grossly false" was not covered by the expert opinions uttered on the Internet. In our dataset, seven p-stories in 2020 and one p-story in 2021 treat the Drosten preprint, its implications or the scandal around it as p\_in\_focus without evoking any reference to a broader, more abstract preprint genre. In 2021 a story about "Science Journalism" in ZON critically picked up on this example:

The pseudo debate launched by Bild… exemplifies how a yellow press paper wants to misunderstand the heart of research, and how it can scandalize a fake controversy with the help of alleged counter-expertise, because lay people can hardly judge such cases, given the high degree of complexity.

More than half of the non-corona p-stories focus on preprints (62% total in Table 2a), compared to only 23% of corona p-stories (total in Table 3b). Stories focussing on preprints more often contain uncertainty framings when also covering corona topics (48% in Table 3a) compared to when they cover non-corona topics (14% in Table 3b). No such difference exists for timeliness or access, confirming the overall observation that preprints are generally framed more controversially in corona p-stories. That 94% of non-corona p-stories marked as genre are also marked as p\_in\_focus (Table 3b) is a result of the abovementioned prepandemic genre discussion in 2018 and 2019, in which preprints were discussed as part of a changing academic publishing system. In contrast, the finding that only 36% of non-corona p-stories coded p\_in\_focus are also coded any\_name is linked to the already discussed finding that the majority of non-corona p-stories that use preprints as sources (min\_ref\_1) do not flag their sources as preprints.

The existence of p-in-focus stories is again an indication that preprints are a newsworthy topic in the German media, both as individual publications (sources) and as an increasingly important genre in science publishing.

#### **Conclusion and discussion**

Guided by the general question of how preprints are covered and discussed in the media, we used a novel approach that is not only based on a large-scale dataset, but also aggregates the results of an in-depth qualitative analysis. In more concrete terms, we analyzed 1,006 news stories mentioning preprints (pstories), identified in a sample of 390,942 stories issued by seven major German news outlets from 2018 to 2021. Our study exceeds previous quantitative analyses by using a text-mining approach that is based on various concepts and synonyms for preprints and thereby is not dependent on the availability of hyperlinks or URLs to preprint servers. As the outbreak of COVID-19 in January 2020 occurred in the middle of our study period, we were able to compare the framing of preprints in two groups, corona p-stories and non-corona p-stories. Our qualitative analysis produced 12 codes of interest relating to the five core concepts: 1) framing, 2) naming, 3) referencing, 4) genre, and 5) focussing.

A first and expected finding was that the number of p-stories rose steeply with the beginning of the pandemic, mirroring the increase of preprints in the dissemination of COVID-19 research. The absolute majority of p-stories in our sample relate to corona reporting. The number of non-corona p-stories remains constant over time.

Second, preprints are framed both in terms of the pros and cons ascribed to them in the broader discourse: scientific uncertainty on the "con-side" and timeliness and access on the "pro-side." Among the two pro-framings, timeliness was always much more salient than access for any cross-comparison of categories/codes. While the rates of combined pro-framings were subject to annual variations between 17% (in 2019) and 31% (in 2018), we found a more distinctive increase of uncertainty framings in corona p-stories compared to non-corona p-stories. Although uncertainty framings generally increased in both groups over time, they more than doubled in non-corona p-stories from the year 2020 to the year 2021, suggesting that the pandemic led to an overall more cautious attitude towards preprints, even in the case of non-pandemic related topics.

Third, we found a variety of naming practices in relation to preprints, and suggested an analytical distinction between naming and framing. Unlike Fleerackers et al. (2021), we did not interpret the name "preprint" as an uncertainty framing device, arguing that the name itself does not directly invoke uncertainty. We did, however, interpret the German translation "Vorab-Veröffentlichung" as a timeliness framing device, because the prefix "vorab" very specifically means "in advance" and is, ironically, commonly defined by the example sentence "the press has been informed in advance."7 Both our quantitative and qualitative analysis revealed that naming practices are still evolving in the German media. The English name "Preprint" has been used more often over time and seems to gradually replace its German translations.

Fourth, we find that preprints are reported and discussed both as sources and as a genre of academic publishing. In the majority of cases, p-stories reference concrete preprints to state and/or support claims or arguments, and almost always p-stories then contain enough information so that the interested reader can, in principle, trace the original preprint, for example via a hyperlink. When addressing preprints as a genre, p-stories often educate the reader about the nature of preprints as well as negotiate the pros and cons associated with preprints. During the pandemic we observe a profound change of tone in genre debates. In comparison to pre-pandemic times, uncertainty framings of preprints doubled while framings of timeliness and access remained constant.

 https://www.duden.de/rechtschreibung/vorab

Finally, we showed that preprints sometimes become a key focus of news stories. Such stories do not merely mention preprints or use them as sources to make an argument, rather they are stories about preprints. P-in-focus stories are much more common in areas unrelated to corona. In fact, more than half of all non-corona p-stories are also p-in-focus stories, and most of them were published in 2018 and 2019, covering for example the general role of preprints in physics or other disciplines. We interpret the existence of p-in-focus stories as an indication that preprints have become a newsworthy topic in the German media, both as individual publications (sources) and as an increasingly important genre in science publishing.

On the basis of these findings, we'd like to discuss the following two points. First, we believe that the use and coverage of preprints present a real challenge to (science) journalism. In science, early access to unreviewed results can be important to drive cutting-edge scientific debates, especially debates in which the involved scientists are able to judge the quality of these results on their own. Journalists, in contrast, are not generally capable of assessing the quality of unreviewed publications, but at the same time they are competing with each other for the latest news and sources. This creates a situation where journalists may become tempted to cite preprints even when their results are of poor quality and thus potentially dangerous when used to inform policy and practice.

Our data shows that some journalists frame preprints in terms of uncertainty, timeliness, and access, and also that uncertainty framings seem to become more salient over time. However, many journalists do not discuss the pros and cons of preprints, and an alarmingly large share of journalists does not even mention that they use or talk about preprints. Describing preprints merely as "studies" or "publications" is dangerous, because it suggests a higher level of scientific certainty as should be associated with preprints. We therefore prompt journalists to become more aware of the differences between peer-reviewed journal articles and preprints. More generally, journalists should acknowledge that there is a palette of scientific genres—both within the realm of peerreviewed publications and within the realm of unreviewed academic publications—each associated with different (epistemic) advantages and disadvantages. Understanding these differences is a prerequisite for a responsible and transparent communication of research to non-scientific audiences.

Second, in using preprints as sources and in publicly negotiating the pros and cons of preprints (or failing to do so), journalists may influence scientific preprint practices in direct and indirect ways. The politicization of preprints, especially when resulting in personal attacks of their authors, as in the case of the Drosten study, can intimidate and de-incentivise scientists to make their

findings publicly available prior to official peer review. For example, a statistician who critically commented Drosten's first manuscript on a preprint server confessed: "Had I known that Bild reads this sentence, I had definitely not written it [sic]" (2020, SPON). When, in addition, policymakers argue and make decisions on the basis of preprinted results, scientists increasingly fear that others may try to hold them responsible for political consequences of such decisions. In this regard, it is no surprise that we increasingly see warnings issued by preprint servers directly aimed at journalists and policymakers.

To better understand the changing role of preprints in journalism, especially since the explosion of preprints during the pandemic, further research is needed. One direction would be to interview journalists and scientists on the paradoxes and interrelationships that we just elaborated on. How do journalists reflect their use of preprints? How do scientists view the impact of media attention to preprint publishing? Another direction could be to link citation context analysis in the media with bibliometric data from databases like Web of Science or Scopus. This could enable the analysis of citation latency as well as, potentially, the causal modeling of preprint reporting in the media on citation behavior of scientists. In addition, empirical analysis of how different framings impact on the understanding by readers can enrich the literature on narrative framing.

#### **References**


to acquire science knowledge. *Journal of Science Communication, 20*(3), A08. https://doi.org/10.22323/2.20030208

Chiarelli, A., Johnson, R., Pinfield, S., & Richens, E. (2019). *Preprints and scholarly communication: An exploratory qualitative study of adoption, practices, drivers and barriers* (8:971). F1000Research. https://doi.org/10.12688/f1000research.19619.2

Cobb, M. (2017). The prehistory of biology preprints: A forgotten experiment from the 1960s. *PLOS Biology, 15*(11), e2003995. https://doi.org/10.1371/journal.pbio.2003995

Colavizza, G., Costas, R., Traag, V.A., Eck, N.J. van, Leeuwen, T. van, & Waltman, L. (2021). A scientometric overview of CORD-19. *PLOS ONE, 16*(1), e0244839. https://doi.org/10.1371/journal.pone.0244839

Craig, G. (2016). Reclaiming slowness in journalism. *Journalism Practice, 10*(4), 461-475. https://doi.org/10.1080/17512786.2015.1100521

Dijk, T.A. van (1988). *News as discourse*. Routledge. https://doi.org/10.4324/9780203062784


Dunwoody, S. (2014). Science journalism: Prospects in the digital age. In *Routledge handbook of public communication of science and technology*. Routledge.


Heimstädt, M. (2020, April 3). Between fast science and fake news: Preprint servers are political. *Impact of Social Sciences*.

https://blogs.lse.ac.uk/impactofsocialsciences/2020/04/03/between-fast-science-andfake-news-preprint-servers-are-political/

Horbach, S.P.J.M. (2020). Pandemic publishing: Medical journals strongly speed up their publication process for COVID-19. *Quantitative Science Studies, 1*(3), 1056-1067. https://doi.org/10.1162/qss\_a\_00076


Karlsson, M., & Sjøvaag, H. (2018). Hyperlinks and linking practice. In *The international encyclopedia of journalism studies* (S. 1-5). John Wiley & Sons, Ltd. https://doi.org/10.1002/9781118841570.iejs0231

Kreps, S.E., & Kriner, D.L. (2020). Model uncertainty, political contestation, and public trust in science: Evidence from the COVID-19 pandemic. *Science Advances, 6*(43), eabd4563. https://doi.org/10.1126/sciadv.abd4563

Larivière, V., Haustein, S., & Mongeon, P. (2015). The oligopoly of academic publishers in the digital era. *PLOS ONE, 10*(6), e0127502. https://doi.org/10.1371/journal.pone.0127502

Latour, B. (1988). *Science in action: How to follow scientists and engineers through society* (Reprint Edition). Harvard University Press.

Le Masurier, M. (2015). What is slow journalism? *Journalism Practice, 9*(2), 138-152. https://doi.org/10.1080/17512786.2014.916471

Lewenstein, B.V. (1995). From fax to facts: Communication in the cold fusion saga. *Social Studies of Science, 25*(3), 403-436. https://doi.org/10.1177/030631295025003001

Matthias, L., Fleerackers, A., & Alperin, J.P. (2020). Framing science: How opioid research is presented in online news media. *Frontiers in Communication, 5*. https://www.frontiersin.org/articles/10.3389/fcomm.2020.00064

Mede, N.G. (2022). Legacy media as inhibitors and drivers of public reservations against science: Global survey evidence on the link between media use and anti-science attitudes. *Humanities and Social Sciences Communications, 9*(1), Art. 1. https://doi.org/10.1057/s41599-022-01058-y

Moore, C.A. (1965). Preprints. An old information device with new outlooks. *Journal of Chemical Documentation, 5*(3), 126-128. https://doi.org/10.1021/c160018a003

Nelkin, D. (1995). *Selling science: How the press covers science and technology*. W.H. Freeman & Co Ltd.

Pan, Z., & Kosicki, G.M. (1993). Framing analysis: An approach to news discourse. *Political Communication, 10*(1), 55-75. https://doi.org/10.1080/10584609.1993.9962963

Parviainen, J. (1998). *Bodies moving and moved: A phenomenological analysis of the dancing subject and the cognitive and ethical values of dance art*. Tampere University Press.

Riles, J.M., Sangalang, A., Hurley, R.J., & Tewksbury, D. (2015). Framing cancer for online news: Implications for popular perceptions of cancer. *Journal of Communication, 65*(6), 1018- 1040. https://doi.org/10.1111/jcom.12183

Soderberg, C.K., Errington, T.M., & Nosek, B.A. (2020). Credibility of preprints: An interdisciplinary survey of researchers. *Royal Society Open Science, 7*(10), 201520. https://doi.org/10.1098/rsos.201520

Stroobant, J., & Raeymaeckers, K. (2019). Hypertextuality in net-native health news: A quantitative content analysis of hyperlinks and where they lead to. *Journal of Applied Journalism & Media Studies, 8*(3), 367-385. https://doi.org/10.1386/ajms\_00007\_1

The Associated Press. (2019). *The associated press stylebook 2019: And briefing on media law* (New Edition). Basic Books.


Tuchman, G. (1980). *Making news*. Free Press.

Wang, P., & Tian, D. (2021). Bibliometric analysis of global scientific research on COVID-19. *Journal of Biosafety and Biosecurity, 3*(1), 4-9. https://doi.org/10.1016/j.jobb.2020.12.002

Weingart, P. (1998). Science and the media. *Research Policy, 27*(8), 869-879. https://doi.org/10.1016/S0048-7333(98)00096-1

Weingart, P., & Guenther, L. (2016). Science communication and the issue of trust | JCOM. *Journal of Science Communication, 15*(5), C01. https://doi.org/10.22323/2.15050301

### Max Brede, Athanasios Mazarakis, and Isabella Peters **4. What drives researchers to look up research publications they found in the news?**

**Abstract:** External science communication uses media and other means, such as news reports on scientific publications, to produce awareness and understanding of science and its results. Scientific publications that were featured in the news are linked to higher citations and altmetric-counts when compared to similar unfeatured articles. So far, the question about the relationship between attributes of scientific publications, their mentions in a news report, and their effect on researchers' decision to look up a scientific publication remained unanswered: a research gap this study attempts to fill. First, we conducted a threephased variation of a Delphi survey to generate a selection of attributes that experts deem relevant for evaluating scientific publications. Then the attributes were discussed with a focus group and optimized for a large-scale online conjoint study with 642 respondents. Statistical analysis revealed that attributes which indicate expert opinion and methodological quality are the major drivers behind looking up scientific publications mentioned in news reports. This finding underscores that forms of external science communication and the highlighting of particular publication attributes positively affect the awareness of scientific publications that are also positively related with a publication's citation counts.

**Keywords:** conjoint study, Delphi study, news article, science communication

### **Introduction**

Science communication is defined by Burns et al. (2003) as the use of media and other means to produce awareness, enjoyment, interest, opinions, or understanding of science and its aspects. In particular, being aware of new scientific findings is an essential part of every scientist's daily life, as their work must always be up to date. In a study that examined scientists' search and reading behavior, Tenopir et al. (2019) found that scientists report newspapers as an important resource of scholarly information. This highlights the important role of "external science communication" or the communication of scientific knowledge by individuals who are not necessarily part of a scientific community (Dernbach et al., 2012). An example of this type of communication is a newspaper report in which a journalist reports on a research publication.

Scientific publications featured in the news have been shown to receive more citations (i.e., Anderson et al., 2020; Dumas-Mallet et al., 2020; Fanelli, 2013) and higher altmetric counts (i.e., Bowman & Hassan, 2019; Lemke, 2020). Most often, this effect is discussed to be attributable to one of two possible mechanisms or a mix of both (see also Chapter 5 in this book; Lemke, 2022). The first is the so-called earmark hypothesis (Kiernan, 2003) which attributes the observable advantage to a qualitative difference in publications mentioned in news reports because the publication responds similarly well to the selection strategies of researchers and journalists alike. The idea is that researchers search for and cite similar publications (and publication attributes) like journalists, resulting in higher citation counts, regardless of the increased reach and larger audience resulting from the non-academic news report. Here, one may assume that researchers and journalists have similar mental concepts about and selection strategies for the "newsworthiness" or "relevance" of scientific publications. This thesis of inherent qualities of research publications driving citations and other forms of attention is backed up by findings from a different scenario. Breuer and colleagues (2022) used retrieval test collections to compare relevance judgments for scientific publications with their citation rates, although they have not investigated in detail the role of publication qualities for relevance decisions. They showed, though, "that documents that receive a relevance rating are more likely also to be highly cited" (Breuer et al., 2022, p. 2470) and that they receive higher altmetric attention scores. This connection is unidirectional: highly cited documents are not necessarily (more) relevant for a search task.

In contrast to this pure attribution to the publication's attributes, the publicity hypothesis states that the advantage in citations can be linked to the additional reach gained by journalistic reporting. A strong indicator of this connection is the study by Phillips et al. (1991). The authors compared a sample of research publications featured in the New York Times to one that was meant to be featured in news reports but was not, due to a strike. The authors reported that the publications featured in the unpublished edition of the newspaper did not perform any better than comparable, not-featured articles. This citation advantage associated with a mention in the New York Times was replicated by Kiernan (2003), who was also able to show that the reported effect is not specific to this prestigious outlet.

Both hypotheses are not mutually exclusive. It could also be argued that the journalistic landscape and its interactions with academia have changed drasti-

cally since these explanatory models were formulated. Examples of these changes are the rise of social media and its usage by scholars (Lemke et al., 2019).

Furthermore, neither hypothesis goes into the specifics of which attributes of a scientific publication lead to the observed advantages. However, it is reasonable to assume that certain attributes might result in different effects in both models, and the way agents interact with scientific publications might depend on different attributes. One could, for example, expect that journalists choose publications depending on specific criteria, like the "newsworthiness" of their topic that is not necessarily equal to or related to newsworthiness in science (see also Chapter 2 in this book). In contrast, researchers may choose to look up the publication and cite it, depending on its relevance to their research, because of disciplinary norms, acquaintance with the authors, or because of many other reasons that are often not explicit (Cronin, 1981; Garfield, 1962; Tahamtan & Bornmann, 2019).

In fact, such selection and decision-making processes are driven by multiple criteria that are often intertwined and prioritized according to the actual topic or situation and the role of the person who selects. For example, it has been shown for an intermediary institution between science journalism and science, the Science Media Center Germany (Broer & Pröschel, 2021), that it selects publications by acknowledging four major sets of criteria: a) journalistic criteria, such as reach, relevance for the public, urgency; b) science-internal criteria, such as quality of the journal, sample size, used method, number of authors; c) strategic criteria, such as the impact on public discourse or on agenda-setting; and d) organizational criteria, such as availability of experts and editors in the institution. Those findings highlight that science-internal and science-external selection strategies from researchers and other actors can either reinforce each other (as in the earmark hypothesis) or shed light on what is valued by the different actors of the science communication system.

To study the specifics of these interactions of internal and external scholarly communication, citations are of particular interest since they should be sensitive to publicity and earmark effects. In most cases, bibliometric citation analysis is concerned with the characteristics of the publication, its authors, and the journal in which the publication appeared. In this regard, most of these studies are based on post hoc analyses of publications whose characteristics and citations were used for analysis (Tahamtan & Bornmann, 2018).

In contrast, Tenopir and colleagues (2011) conducted a "conjoint analysis" style experiment. Conjoint analysis, also known as "discrete choice experiment" (Louviere et al., 2010) or "choice-based conjoint analysis" (Backhaus et al.,

2015), attempts to describe an entity in terms of its attributes and to identify the attributes promising the most (partial) utility of that entity to a user.

This type of survey design is primarily used in marketing and consumer research studies of latent preferences (Backhaus et al., 2015; Louviere et al., 2010). The goal is always to determine which attributes of an entity influence a participant's preferences in which magnitude. The procedure is based on a part-worth model that defines an option's value or utility as the sum of its attributes' partworth utilities (Louviere et al., 2010). Conjoint analysis has successfully been used in a variety of settings, i.e., to test biases in the choices of healthcare stakeholders (Crabtree et al., 2022), to evaluate information leak severities (Koguchi & Maeda, 2022), or to examine the perception of privacy issues in virtual reality technology of German consumers (Schuir et al., 2022).

Tenopir and colleagues (2011) used a conjoint analysis-based survey to identify the most important features of a publication that make a potential reader want to read it. To do this, the authors examined the three attributes "author reputation," "journal prominence," and "online accessibility of the publication." The authors concluded that the accessibility of the publication is the most important of the three attributes. This was followed by the reputation of the authors and, finally, the type of journal as the least important attribute. In an additional choice experiment, the authors found that the "topic of the article" played by far the most important role. Other than that, the results of the conjoint analysis were replicated. Since a realistic choice between two publications is likely to cover similar topics, Tenopir and colleagues (2011) conclude that the three attributes considered in the experiment are of most use.

Another conjoint approach to analyze attributes influencing citations was conducted by Lemke and colleagues (2021). Their goal was to determine which bibliometric indicators are most helpful for readers when deciding whether to cite the article. They concluded that citation counts and the journal impact factor are the attributes generating the highest utility.

The research described so far relates to the scholarly reading and use of scientific literature. The state of the art of research dealing with the interaction of internal and external science communication mainly refers to the mention of scientific publications in news media in terms of an altmetric perspective. These altmetrics-centered studies are mainly concerned with the meaning of individual altmetrics (Haustein et al., 2015) or the ways to collect them, e.g., Kousha and Thelwall (2019), i.e., more with their use than with their creation. Corresponding studies mainly deal with other, non-news media-related data, such as the use of social media by scientists (Van Noorden, 2014). However, a more detailed analysis of the influences on the effect of external science communication and,

more specifically, news reports on subsequent citations of original publications beyond the theoretical consideration described above is lacking.

The empirical approach described below represents an attempt to gather initial indications about this problem. To the authors' knowledge, this study is the first attempt in this direction. So far, no detailed analysis of the interplay of publication attributes reported by news and their effect on later reads by researchers and citations of publications has been conducted. Therefore, the following reasoning underlies the study design: original research publications expose certain qualities (e.g., relevance, quality, rigor, innovation, topic), which can be translated to or operationalized via certain attributes that reflect those qualities (e.g., sample size, number of citations) and that can be linked to additional qualifiers allowing for nuance (e.g., large, groundbreaking). The attributes respond to researchers' mental selection strategies and drive decisions. The attributes can be mentioned in news media reports and other forms of external science communication that report on the original publication. We assume that 1) external science communication raises researchers' awareness of research publications (see also publicity hypothesis; Phillips et al. (1991) and Chapter 5 in this book) and 2) mentioning attributes in external science communication increases the likelihood of researchers' looking up (and then citing) the original publication.1 The research question we attempt to answer in an exploratory examination of the aforementioned interplay focuses on the second part of our basic assumption: which are the key attributes of scientific publications whose inclusion in news reports might be beneficial to scientists in deciding to look up the publication that is being reported on?

If attributes of that kind could be found and their effect can be quantified, they could support an understanding of the extent to which news media influence scholarly citations. Since the research question aims at the implicit valueattribution of the reading scientists, a conjoint analysis was constructed based on the experiments by Tenopir et al. (2011) and Lemke et al. (2021).

The contribution of this article is threefold. First, we present a collection of expert-verified attributes of scientific publications that are deemed relevant (via a modified Delphi survey (Pollitt et al., 2016) when deciding whether a publication will be looked up (and subsequently cited) or not. The attributes served as

 **1** This is a simplification of the decision-making process, of course. We acknowledge that not all research publications that are looked up will be cited subsequently. However, we assume that awareness of an original publication is a necessary prerequisite for citation and that awareness can be induced via several mechanisms, e.g., formats of external science communication, search results lists, reference lists.

stimulus material in a conjoint experiment, which led to an empirically validated ranking of those attributes. This sheds light on those characteristics of research publications that drive look-up decisions the most. The third contribution comprises an evaluation of the congruence of implicit and explicit decisionmaking behavior of the subjects participating in the conjoint experiment: are they aware of their selection strategies and the publication's attributes they prefer most?

### **Methods**

The conjoint study was constructed in multiple steps. The first step was the generation of relevant attributes and the stimulus material necessary for the online study. A Delphi-method (Gordon, 1994) oriented approach was carried out in three phases to generate an expert-judgment-based selection of attributes.

These attributes were then aggregated and specified in attribute levels to form stimulus material, which another group of experts again validated. A preliminary conjoint study was then conducted using this material to test it for the target audience. Finally, the results from this conjoint pre-study were used to optimize the design for a large-scale online conjoint study. This design process is depicted in Figure 1.

#### **. Delphi pre-study**

A central step in conducting a conjoint analysis is the decision on attributes to be used for the choice sets. Since there are no prior studies examining the direct influence of the mention of specific publication characteristics in news media reports on the latter citation rates, attributes reported to influence the impact of a publication seem to be a good first selection on which to base the choice set (Tahamtan et al., 2016; Tahamtan & Bornmann, 2018; Tenopir et al., 2011, 2019). This decision was founded on our assumption that a greater impact moderated by news media distribution, as postulated in the publicity hypothesis, would be facilitated by mentioning attributes that are relevant to a scientific audience.

**Figure 1:** Overview of the study.

Considering the large variety of discussed factors influencing the impact of a publication, we first conducted a three-phased Delphi pre-study. The goal of this approach was to reduce the broad array of possible attributes to an easierto-handle subset that could be presented in a conjoint setting. A focus groupbased approach oriented on Pollitt et al. (2016) was chosen to do this. For the focus group's convenience, a LimeSurvey (LimeSurvey Project Team/ Carsten Schmitz, 2012) adaption of the Delphi method (Gordon, 1994) was implemented while trying to reach a consensus.

The main advantage of this Delphi-oriented method is its proven efficacy in reaching a productive solution based on small group discussions. Bloor et al. (2015) conclude that a sample of four experts can already produce useful results, given the balanced composition of this group. Similar group sizes of eight to 12 subjects (Robson & McCartan, 2016) or less (Krueger, 2014) have also been reported to work. We aimed to match these sizes while having a diverse group by recruiting seven scientists from different fields for our focus group (two from information science and one each from business informatics, computer science, neuroscience, physics, and criminology). We used 64 attributes listed and classified by Tahamtan et al. (2016) and Tahamtan and Bornmann (2018) to ask our focus group whether the presented attributes might be relevant when looking up research publications mentioned in a news report. The complete list of the 64 attributes can be found in the Appendix.

**Figure 2:** Results of the first and second phases of the Delphi pre-study. The importance values based on the ranking are calculated by setting the first rank to one and decreasing the value in steps of 1/n, where n is the number of ranks assigned. Items that were not ranked were assigned a value of 0. The items were presented in German.

The focus group reported their decision on a binary scale consisting of "not relevant" and "relevant" labels. At the end of the first Delphi phase, the focus group had the chance to list additional attributes they found to be missing in the initial list. Of the initial 64, 54 attributes with at least one vote for relevance were available for the second Delphi phase. This attribute collection was supplemented by 12 additional attributes the participants mentioned to be missing. All initial attributes, the additional attributes, and the amount of "relevant" votes by the focus group are available in the Appendix. This collection marked the end of the first Delphi phase and led to the second Delphi phase.

The resulting list of 66 attributes was presented to the experts in the second Delphi phase, this time with the task to first rate the importance of each attrib-

ute on a seven-point Likert scale and then additionally to rank up to ten attributes as the most important. This approach resulted in a selection of six attributes that were ranked and rated as the most relevant, namely "methodological quality," "importance of the research," "journal or conference as an outlet," "type of research design," "statistical methodology" and "sample size." The distribution of rankings and ratings is shown in Figure 2. In addition, the results of all ratings and rankings are available in the Appendix.

In the third phase of the Delphi pre-study, the participants were asked to confirm the six selected attributes and explain their reasoning. The confirmation was collected online and within a focus group setting. This was done to avoid high scoring items being too broad in their possible interpretations. "Methodological quality," as an example, can be understood in different ways, depending on the background of the expert.

All participants of the focus group agreed to the six selected attributes. This result concluded the end of the Delphi pre-study and led to the specification of the attribute levels for all six confirmed attributes.

#### **. Specification of attribute levels**

Because conjoint analyses are based on entities defined by combinations of levels of attributes, the selection of attributes had to be appended by appropriate levels in the next step. To do this, the explanations from the third phase of the Delphi pre-study and a literature review (Bhandari et al., 2007; M. Callaham et al., 2002; M. L. Callaham et al., 1998; Craig et al., 2007; Farshad et al., 2013; Figg et al., 2006; Kulkarni et al., 2007; Miettunen et al., 2002; Miettunen & Nieminen, 2003; Nieri et al., 2007; Patsopoulos et al., 2005; Willis et al., 2011) were used to generate up to four levels for each of the attributes, which were then combined into sentences to form mock-up-articles as the final choice set basis and thus the basis for the material of our conjoint study. The resulting levels for each attribute are displayed in Table 1. The number of attributes and levels thereof are based on the average conjoint designs, as reported by Marshall et al. (2010). More details about the attributes can also be found in the Appendix.

The complete mock-up design and the plausibility of the different values in Table 1 have been checked by representatives of the Science Media Center Germany who were partners in the MeWiKo research project (MeWiKo, n.d.) and who brought in journalistic expertise.


**Table 1:** The components of the choice sets resulting from the Delphi pre-study. The original choice sets were displayed in German.

#### **. Conjoint-design**

The conjoint analysis was then conducted using a lab.js-based (Henninger et al., 2019) online questionnaire, asking the participants to rank three mock-up news reports per trial. This was done by providing an instructional text at the top of each page that instructed the participants to arrange the mock-ups in the order they evoked curiosity to look up the original scientific publication. An example of such a mock-up news report can be seen in Figure 3.

The attributes' presentation was conducted with mock-up reports using a simple list of the attribute levels due to the results of the Delphi pre-study. Some of the experts in the Delphi pre-study reported issues in keeping the news media focus of the study in mind. The presentation in the form of a news report was meant to keep the context present.

The whole study consisted of 12 trials, which were chosen based on Fedorov's exchange algorithm (Fedorov, 1972). This algorithm optimizes the information gained by an experimental design by choosing the design options that maximize the marginal effects.

Participants were invited to the study via e-mail in two steps, using a mailing list of economics researchers provided by the ZBW - Leibniz Information Centre for Economics. The first step intended to test the design on representatives of the target population to see whether they encountered any issues with the study material. Additionally, the first step was conducted to test the selected choice sets on their viability. The second step was then adapted to the lessons learned from the first step. Then the study was fully rolled out to gather and analyze the data regarding the research question, as depicted in Figure 1.

**Figure 3:** Mock-up news report as used in the conjoint study.

#### **. Conjoint step 1 – pre-study**

The first step consisted of 2,000 e-mail invitations sent out between February 17 and March 15 2021. Of the 2,000 invited researchers, 156 accessed the study. Of those, two were excluded due to stating nonsensical employments, eight were excluded for taking more than ten minutes for one trial, and one subject was excluded for indicating not working scientifically.

Most of the 145 participants indicated working as a professor (42.86%), followed by Ph.D. students (20.41%) and research assistants (19.31%). The largest group of participants reported their research field as economics (40%), followed by business studies (34.48%). After finishing the 12 trials, the participants had the opportunity to give verbal feedback. Two independent raters categorized these verbal answers and achieved a satisfactory inter-rater reliability of αKrippendorff= .742 (cf. Krippendorff, 2004). These verbal answers were then used to adapt the study for conjoint step 2, more precisely, the actual data collection and leaving the pre-study phase. Some of these categories are now presented in detail.

The category "contradiction," which was by far the most frequently assigned by the raters (eight out of 25), was to be assigned in cases where the participants indicated a contradiction in the stimulus material, for example, as stated in the following comment: "I wonder how a study that uses an inappropriate statistical approach can nevertheless be rated as 'methodically well conducted.'"

The choice sets were redesigned to address this problem in the second conjoint step so that contradictions no longer occurred. This change was implemented because many participants described the contradictions as rendering the choice sets nonsensical. This change in design resulted in a high correlation between the attributes STATISTICS and QUALITY. Since high correlations between predictors render conditional logistic regressions unsolvable, one of these attributes must be ignored in the final statistical model.

Besides the remark about a contradiction, criticism of the length of the study and remarks about decreasing concentration were the most frequent type of feedback (five out of 25). A typical example of this type of criticism is the feedback "It is extremely difficult to stay concentrated with the very similar texts and the constant repetition." This criticism was not addressed in the second step because a limitation of statistical power was weighted more heavily than the possible effects of fatigue. However, to control whether selection decisions due to fatigue happened solely via heuristics, an item was added at the end of the questionnaire in which such heuristics were queried by asking the participants to state attribute-based decision-heuristics they were aware of.

These heuristics were tested by asking the respondents to put the attributes in the order in which they thought they influenced their decisions.

The third most frequent type of comment (four out of 25 each) was the praise of the design ("It is nice to see conjoint being used") and criticism of the stimulus material. There was especially repeated criticism about the vertical arrangement of the mock-up news reports, which resulted in annoying scrolling, as can be seen in this comment: "The display was unfortunately not optimal on my computer. Smaller images that could have been dragged down or up, if necessary, rather than to the right would have been better. Could unfortunately never see all three at once." To address this criticism, the design for the second conjoint step was changed to display mock-up news reports and categories horizontally on large screens.

We also received some comments from a few subjects who did not read the texts thoroughly but only focused on the highlighted passages. Others indicated that their decisions had been made based on a subset of these without attention to the overall context. Although these were only a few subjects, we took these indications seriously and used a manipulation check to control this aspect in the second conjoint step.

#### **. Conjoint step 2 – data collection**

Data collection for the second step began on May 11, 2021, and ended on June 9, 2021. To recruit subjects for the second step, invitations were sent to 6,000 previously unused e-mail addresses from the list provided by the ZBW - Leibniz Information Centre for Economics on May 11, and reminders were sent on May 25. During the data collection period, 728 potential participants started the study, of which 36 were excluded for trials with completion times over ten minutes, ten for non-scientific employment, and 185 for not completing all 12 trials. The remaining participants took a mean of 43.9 seconds to complete a trial, with a standard deviation of 37.2 and a median of 35 seconds.

The 497 participants remaining were mostly professors (43.26%), followed by doctoral students (25.96%). As in the first step, most of these participants came from the fields of economics (34.81%) and business administration (29.18%), followed by macroeconomics (10.66%).

### **Results**

In order to evaluate the data collected in the second conjoint step, two conditional logistic regression models were calculated, using the participant's arbitrary ID and the question number as stratum (this procedure is oriented on the method described in Aizaki and Nishimura (2008)). The first regression model analyzed the first rank priority, and the second regression model the second rank priority. However, as already noted, one of the two attributes, "methodological quality" (QUALITY) and (STATISTICS), must be excluded from the evaluation. Since the former was rated as much more critical in the Delphi-oriented phases held in advance, the decision for exclusion fell on "adequacy of the statistical procedure." Another argument for this exclusion are textual comments by participants, such as the following: "I come from a discipline that does virtually no quantitative or qualitative empirical work." The fact that "methodological quality" is a more flexible term and can be interpreted by many disciplines as relevant to themselves is another argument for preferring this attribute; "adequacy of statistical procedure," in contrast, is only relevant to disciplines that generate inference based on statistical models.

The results of the conditional logistic regressions are shown in Table 2. When looking at the estimated utilities, it is noticeable that "methodological quality" seems to be far more relevant for the second priority than for the first rank.

**Figure 4:** Relative utility of the presented attributes as the range between the highest and lowest level relativized by the sum of these ranges.

**Table 2:** Results of the two separate conditional logistic regression models. The first column indicates whether the decision for the first level over the others or the decision for the second level over the third was modeled.



The attribute that evoked the most interest in looking up the original publication seems to be the judged "importance" for the research area. This impression becomes even clearer when the relative utilities are considered (Figure 4). An interesting observation is that the three most important attributes for the first level of priority were those that were linked to an expert judgment in the mockup news report (IMPORTANCE, QUALITY, and DESIGN).

Regarding the second level of priority, it is noticeable that with "methodological quality" and "importance of the research area" extraneous judgments also weighed heavily in the decision. "Sample size," which had the least influence on the first level of priority, is also of great importance for the second level of priority, now achieving the second-highest relative utility. It should be noted that the phrase "large sample for the research area" can also be interpreted as an extraneous judgment, just like the other essential characteristics, but this was not our initial intention.

Another issue investigated in the second conjoint step of the study was whether the participants followed a conscious heuristic when conducting the trials. A question was included as a reaction to many participants of the first conjoint step, remarking that they did not read the texts attentively but only paid superficial attention to the highlighted text passages. They further indicated that their decisions had been made based on a subset of these without attention to the overall context. As already mentioned, this comment occurred particularly in combination with complaints about effort and fatigue resulting from conducting the study.

**Figure 5:** Mean scores of the attribute ranking asking for conscious judgment heuristics. Since the participants had the option to rank a subset of the attributes, the assigned ranks were transformed so that a score of 1 indicates a rank of 1, decreasing by 1/n for each of the n indicated ranks.

When examining the statements on conscious judgment heuristics obtained via a drag-and-drop ranking, it is noticeable that this does not seem to fully coincide with the findings obtained from the logistic regression (Figure 5). Here, the respondents were asked to rank the attributes: "Did you follow conscious decision rules while evaluating the items? If yes, please sort the presented aspects in descending order of importance for your decision."

"Presentation form" was one of the least important attributes in both response formats, and high relevance was attributed to "importance" of the research area. Nevertheless, overall, the structure of the responses is different; for example, "methodological quality" was ranked differently compared to its inclusion in the utility estimate. Still, it can be assumed that the results are due to the overall effects of the mock-up news reports and not to deliberate decision heuristics, leaving aside the possibility that they result from a lack of understanding about the task at hand. This statement leads to the conclusion and limitations in the following section.

#### **Discussion and conclusion**

We have conducted a conjoint analysis to determine which attributes of scientific publications influence the impact of news media reports on look-up decisions and subsequent citations most. For that, we followed the approach by Anderson et al. (2020) and studied which attributes drive the decision to look up an original research publication mentioned in a popular news report on science. The stimulus design was based on a three-phased Delphi study, resulting in six attributes as the most relevant characteristics for informing a decision to look up an original scientific publication, which were then presented in an online conjoint study. The study's participants – primarily senior scientists who conduct research in economics and business studies – were instructed to rank three mock-up news reports consisting of these six attributes each. This ranking task was repeated 12 times and analyzed using two separate conditional logistic regression models to estimate the utility of each attribute influencing the participants' decision-making.

We have shown clear differences in the utility of attributes used to describe scientific publications in a news report. Also, those attributes drive decisionmaking and future behavior (to look up the article or not) of researchers to a different extent, although the experts of the Delphi study deemed all attributes relevant for look-up decisions.

Overall, the subjects of the study have relied mainly on attributes based on expert opinion when looking up a publication underlying a news report. An indication of the importance of the publication, followed by statements about the methodological quality and the research design, positively influences decision-making towards looking up the original research publication. The selection decisions seem to be made on a case-by-case basis rather than based on general heuristics the subjects have followed, which is similar to selection processes in science journalism (Chapter 2 in this book; Broer & Pröschel, 2021). Since the population consisted primarily of economic researchers, these findings might be highly skewed. This is especially plausible since Lemke (2020) found press releases to mainly reference medical journals. Htoo and Na (2017) found significant differences in attention across disciplines in various altmetric indicators, including news coverage. An attempt to replicate the findings based on a different population would be desirable.

In addition, the conjoint analysis did not explicitly take into account the expert role of the participants. Although we did not observe one, a bias in the self-understanding of the researchers as experts in their respective fields could still be present. This could lead to answers that could be rational and consistent

or professionally-expected (habituated) explanations for their choices, ultimately leading to a social-desirability bias. However, observing participants' lack of consistent heuristics somewhat contradicts that idea.

Another issue is the possibility of complex interactions in the statistical utilities. Since the chosen models are not appropriate to estimate such interactions and the underlying conjoint model is one of independent attributes (Louviere et al., 2010), these possible complications in the interpretation of the results were not considered. Although contradictory combinations of attributes were omitted in the main study, one could still argue for amplification effects in certain combinations of attribute levels, such as quality and sample size.

Additionally, the presentation as a mock-up news report could further influence these effects. If the way the mock-up was presented primed a certain trust in the "experts" mentioned, for example, the "expert's opinion" could interact with the other attributes in other ways than a simple list of attributes would induce. However, this is also possible for real newspaper reports that are also impacted by the newspapers' or the journalists' perceived prestige, as they may serve as indications for the quality of the journalistic reporting. The decision to present the attributes in a mock-up format was made since the focus group consulted for the relevant attributes reported having rated the items as relevant for reading a publication in general, not based on news media. Therefore, to prevent this non-intended issue in the study, the mock-ups were used as the medium of presentation. To examine this possible caveat, one could reproduce the survey without the sentences or by presenting only one attribute at a time.

Another matter is the statistical model used to analyze the utilities. Including all decisions into one holistic ordinal logistic regression model, as described in Allison and Christakis (1994), would have been preferable. However, since the central assumption of ordinality of the criterion in every predictor was not met, the alternative approach of using two separate models, as described in Aizaki and Nichimura (2008), was taken. The resulting two conditional logit models came with the price of repeated testing and a lack of an estimate of the basic utility differences between the first and second rank priority levels. A different design of the attribute levels could alleviate this issue and make a more comprehensive model possible.

Our study is mainly theoretically rooted in the earmark hypothesis (Kiernan, 2003) and the publicity hypothesis (Phillips et al., 1991), which both try to explain higher citation counts for scientific publications covered in the media. However, to fully understand the intertwined relationship of external and internal science communication and how they affect each other, additional

theoretical considerations should be taken into account and used for further experimentation. For example, an explanation is needed to understand why not every research publication mentioned in the news will also receive more citations. Here, theoretical thoughts such as those stemming from the attention economy by Georg Franck may prove valuable, especially since it already focuses on the realm of science and academic reputation (van Krieken, 2019, p. 4). Attention is recognized as both a scarce resource and as a basic need. At the same time, however, attention generates more attention (van Krieken, 2019, p. 5): a phenomenon also described as "success breeds success" or the "Matthew Effect" (Klamer & van Dalen, 2002).

Furthermore, news values (initially called "news factors"), as proposed by Galtung and Ruge (1965), can have a significant influence on the flow of news. So it is not surprising that, among other factors, the factors "frequency" and "unexpectedness" of news can yield higher mentions in media (Galtung & Ruge, 1965). This can also be viewed closely with the relevance theory proposed by White (2011), who argued that authors usually cite research publications to strengthen their claims and that produce the least cognitive effort while retrieving and evaluating them (Breuer et al., 2022). Both arguments may also be applied to publishers of news reports and journalists when selecting original publications to be reported on. By no means is this selection of theories complete, which highlights the need for further quantitative and qualitative research on the overlapping processes and effects of internal and external science communication.

The attributes presented in our study are solely based on a literature review concerning influences on scientific impact. Furthermore, most of the attributes presented are not regularly reported in news reports. Therefore, although our study was supposed to have high external validity, this artificial limitation could present difficulties in interpreting the results. A large-scale natural language processing-based approach could help to test this caveat and to actualize the list of attributes with as realistic attributes as possible.

Overall, the results are promising, especially regarding the discussion on whether the observed effects of news reports and mentioning of certain publication attributes on look-up decisions (and, perhaps, later citations) are based on the research publications alone or on the additional visibility due to media coverage. Since the most valuable attributes were those with an external judgment implied, the additional, thematically, and methodologically classifying information that can be relayed by a news report could be one of the most significant driving factors concerning the impact advantage. This also highlights the effect

of intermediary institutions, such as Science Media Centers, that provide background and expertise to science journalists (Broer & Pröschel, 2021).

Our exploratory results are particularly critical since they bear room for discussion regarding the role of internal scholarly communication and science journalism and their relationship (Broer & Rotgeri, 2021). One may argue that news media coverage poses a threat to internal science communication and the reputation system of science, which – besides strong critique (Hicks et al., 2015) – still heavily relies on citations to research publications. Since we see tendencies confirming both the earmark and publicity hypotheses (see also Chapter 5 in this book), news media and external science communication can serve as gatekeepers or science influencers, who channel attention towards certain scientific topics, authors, etc. – and along with it, may raise or amplify awareness of scientific publications (Klamer & van Dalen, 2002) reflecting all positive and negative effects associated with the theory of attention economy (van Krieken, 2019). Amongst others, future work should apply a large-scale natural language processing-based approach to examine whether news reports that provide contextual information about ascientific publication come with higher citation rates of the original publication.

#### **Acknowledgments**

This study is part of the research project MeWiKo funded by the German Federal Ministry of Education and Research (Grant numbers 16PU17018C and 16PU17018A). We also thank two anonymous reviewers for their valuable feedback on the article.

#### **References**


Dumas-Mallet, E., Garenne, A., Boraud, T., & Gonon, F. (2020). Does newspapers coverage influence the citations count of scientific publications? An analysis of biomedical studies. *Scientometrics*, 1-15.

Fanelli, D. (2013). Any publicity is better than none: Newspaper coverage increases citations, in the UK more than in Italy. *Scientometrics, 95*(3), 1167-1177. https://doi.org/10.1007/s11192-012-0925-0

Farshad, M., Sidler, C., & Gerber, C. (2013). Association of scientific and nonscientific factors to citation rates of articles of renowned orthopedic journals. *European Orthopaedics and Traumatology, 4*(3), 125-130. https://doi.org/10.1007/s12570-013-0174-6

Fedorov, V.V. (1972). *Theory of optimal experiments*.


Krueger, R.A. (2014). *Focus groups: A practical guide for applied research*. Sage publications.

Kulkarni, A.V., Busse, J.W., & Shams, I. (2007). Characteristics associated with citation rate of the medical literature. *PLOS ONE, 2*(5), e403.

https://doi.org/10.1371/journal.pone.0000403

Lemke, S. (2020, December 18). *The effect of press releases on promoted articles' citations and altmetrics*. Metrics 2020: Workshop on Informetric and Scientometric Research (SIG/MET). Zenodo. https://doi.org/10.5281/zenodo.4351360



### **Appendix**

**Table:** All attributes as extracted for the focus group-style interviews. The attributes removed after the first phase are missing rankings and ratings for the second phase. The papers used to determine the attribute levels after the Delphi pre-study are referenced by DOI or PMID.













### Steffen Lemke, Athanasios Mazarakis, and Isabella Peters **5. Path model of the interplay between the promotion and the received attention of research articles**

**Abstract:** Existing systematic analyses of the associations between the visibility that research articles receive within different formats of external science communication (e.g., press releases, embargo e-mails, or news stories) and their later impact metrics are mostly restricted to specific case studies, despite these studies' recurring findings of substantial potential effects. This study aims to give a consolidating and more comprehensive perspective on the interplay between research articles' promotion within press releases and embargo e-mails, their publishing journal's prestige, as well as their received attention in mainstream media, on Twitter, and their academic impact as proxied by citations. To achieve this goal, we use the path analysis method to specifiy models that manifest a range of hypotheses motivated by literature and theory on the relationships that may exist between these variables. We estimate and evaluate our models based on a dataset of 67,581 research articles, which we construct through a combination of empirical data from Web of Science, Altmetric.com, EurekAlert!, and Science Media Center Germany. The resulting model confirms the conformity of the hypotheses we derived from past literature with the large set of empirical observations within our sample. More specifically, our findings highlight the considerable associations between promotion in external science communication and the attention research articles can be expected to receive on both mainstream and social media. The strongest correlations in the model exist between mainstream media mentions and both embargo e-mail promotion (normalized path coefficient of 0.605) and press release promotion (normalized path coefficient of 0.568).

**Keywords:** press releases, embargo e-mails, citations, altmetrics, external science communication, path analysis

### **Introduction**

Citation analysis – i.e., the analysis of citations to scientific publications as indicators for the latter's academic influence – assumes that the likelihood of an article getting cited correlates with relevant inherent qualities, e.g., its scientific rigor, its novelty, or the significance of its findings. While certain studies argue that citations can indeed be useful predictors for such inherent qualities (see Aksnes et al., 2019 for an in-depth discussion), scientometric research has revealed numerous factors without apparent relation to quality that also affect an article's likelihood of being cited. Being informed of their existence and effects is crucial to assess individual citation analysis applications' validity and potential limitations.

In their review of literature examining factors that affect citations, Tahamtan et al. (2016) divide 28 such factors into the categories of paper related, journal related, and author related factors, albeit acknowledging that this selection is not exhaustive. As the three categories identified by Tahamtan et al. (2016) indicate, a large portion of the research focuses on how respective research findings are communicated within the academic sphere, e.g., how decisions regarding publication formats or publication venues affect citations. These decisions can, for the largest part, be considered aspects of internal science communication or scholarly communication, i.e., the communication of research findings primarily targeted at an academic audience. However, another, less analyzed set of issues affecting citations results from how research is featured and processed in channels of external science communication, i.e., in media targeted at stakeholders outside the scientific community.

Considering the scientific journal article as the exemplary unit of observation, such processing by public media may begin even before said article's publication: many scholarly journals regularly disseminate advance information on upcoming issues to science journalists in an arrangement known as an embargo (Kiernan, 1997). In short, this arrangement provides registered journalists with early access to unpublished research findings in exchange for their pledge not to pass on that information before a specified embargo date has passed. The embargo system serves both involved parties well: for the science journalists, embargoed information allows them to timely prepare their coverage on new findings while also providing a certain assurance that other journalistic outlets' respective stories will not leapfrog their own – provided those other outlets do not break the embargo date, of course. For scholarly publishers, the embargo system provides an opportunity to highlight specific publications and topics to the media as well as a certain control over the respective coverage's timing (Kiernan, 2014). As the embargo information given to journalists usually requires prior registration and thus is not openly accessible, it remains difficult to assess how this specific form of promotion affects the attention individual journal articles later receive, let alone their probability of getting cited.

Other types of interventions in external science communication that serve the purpose of directing attention to certain articles are less difficult to track. One of the most common tools used by press officers of scholarly publishers, journals, or research institutes to promote selected publications is the press release, described by Carver (2014, p. 2) as "essentially a short news article written in a journalistic style that explains a newly published scientific result in a common and not too specialized language." While research on the relationship between press release promotion and articles' later citations is rather scarce, some case studies indicate a positive association between the two (Chapman et al., 2007; Fanelli, 2013; Lemke, 2020), although Fanelli (2013) found that this association becomes negligible when controlling for the media coverage of the respective articles..

On the topic of media coverage, a further body of case studies examined how mentions in newspapers affect scientific publications' later citations (Dumas-Mallet et al., 2020; Fanelli, 2013; Kiernan, 2003; Phillips et al., 1991). Phillips et al. (1991) found articles from the *New England Journal of Medicine* (NEJM) to receive significantly more citations if these had been featured in the *New York Times* (NYT) – although this advantage could not be detected for NEJM articles that had been chosen for coverage in NYT issues that in the end were not disseminated due to the NYT being on strike. This finding backs up what is called the publicity hypothesis, which attributes articles' increase in citations associated with press coverage to the concomitant increase in visibility. The earmark hypothesis, on the other hand, explains higher citation counts for press-covered research articles with the assumption that journalistic agents merely apply similar criteria in their decisions on which research to cover as researchers do when deciding which research to cite (Phillips et al., 1991). Kiernan (2003) added to the work by Phillips et al. (1991) by additionally regarding how coverage from 24 daily newspapers and several evening broadcasts of major U.S.-television networks affected citation rates. The author found that the NYT's influence on citation rates is not unique, as NYT coverage did not correlate significantly with citation counts once the author controlled for coverage by other newspapers and television networks. In his study of the association between newspaper coverage and citations, Fanelli (2013) also found regional effects to play a substantial role, as the apparent positive effect of newspaper coverage on citations was stronger for English media than for Italian media, which primarily only affected authors from Italy. More recently, in their analysis of the citation advantages of 162 biomedical association studies reported in newspapers from six specific countries, Dumas-Mallet et al. (2020) found the strength of the observed effects to depend on the influence of the covering newspaper as well as on the number of published press articles. Moreover, they found the positive correlation between newspaper coverage and

citation counts to be most significant for research articles published in journals with lower impact factors.

As the examples of embargo e-mails, press releases, newspapers, and television broadcasts illustrate, the landscape of sources of external science communication with potential effects on research impact is vast, heterogeneous, and at times opaque. Several current developments add to this intricacy: ongoing professionalization of institutes' own research communication, as well as increasing commitments to Open Science and the "third mission" of higher education (Berghaeuser & Hoelscher, 2020), eradicate formerly existing boundaries between scientists, journalists, and public audiences and lead to the establishment of new tools and formats of science communication, many of which enable more direct, bidirectional exchanges of and about research content (Liang et al., 2014). In this vein, an additional layer of complexity results from the increasing digitalization of journalistic media, the advent of social media, and the continually blurring line between these two spheres. In a reciprocal give and take, news stories are posted to and might evoke public discussions on social media platforms like Facebook or Twitter, while journalists also use these platforms to gather news in the first place (Hermida, 2012). Although altmetrics (Priem et al., 2010) provide flexible technical means to track the attention individual research publications attract on various online domains, modelling the relationship between the attention received, for instance, on Twitter, mentions received in the news, and academic citations remains complicated because of such chicken-or-egg dilemmas.

#### **. Research aim and model**

While numerous previous studies put spotlights on specific types of research promotion in external science communication and the subsequent impact metrics of the respective research articles, what is lacking are more comprehensive models that provide entry points for understanding the interdependencies that work between the various interventions made to increase the publicity of research and the attention observable in different spheres of media and academia as a whole. The present analysis represents a step towards closing that research gap. It aims to consolidate the findings from previous case studies, assess their conformity with large sets of empirical data, and extend them by widening the focus to, in this context, under-analyzed forms of research promotion (in the form of embargo e-mails) as well as research metrics (in the form of altmetrics). The findings quantify the degree to which measures of external science communication potentially affect research metrics and provide first steps towards making these degrees comparable, also with a factor from the domain of scholarly communication – journal reputation – whose significance for metrics is well documented (Tahamtan et al., 2016). More substantiated knowledge of such dependencies of research metrics from external factors is necessary to develop more precise descriptions of their meaning, their caveats, and their limitations, enabling more substantiated assessments of their validity within different use cases, e.g., research evaluation. Therefore, our research question can be framed to determine the direct and indirect effects of journal prestige and mentions of scientific publications in various media on later citations. Thus, this study aims to inform the development of a more profound theory on the meaning of various research metrics and to shed light on the complex relationships of attention distribution and channeling that surround and impact research metrics, in order to ultimately increase their usefulness in practice.

Methodologically, this study aims to disentangle the complexity of relationships between various formats of science communication and research impact by formulating and testing path models comprised of several variables that capture particularly relevant manifestations of research articles' impact and uptake in external science communication. We derive hypotheses from literature about the interplay between the attention that research articles receive within press releases, embargo e-mails, mainstream media, social media, their citations within the academic sphere, and their publication venue's prestige and apply the method of path analysis (Regorz, 2021; Streiner, 2005) to see to which degree these hypotheses can be confirmed through the testing of models based on a large set of empirical data. We choose the path analysis method because of suspected multi-level interdependencies between the different indicators to be included in our models and rely on its implementation from the R-package lavaan.1 As an extension of the statistical method of multiple regression, path analysis allows us to test more complex models in which certain variables simultaneously affect and are affected by others.

The model we start our analysis with (Figure 1) is motivated by past empirical research and theory, and is based on six such variables, which we explain in detail after listing them:


 **1** https://cran.r-project.org/web/packages/lavaan/index.html


Our starting model will be fitted to a large set of empirical data, then evaluated for its fit, and afterwards, if appropriate, respecified and reevaluated until no substantial enhancements appear possible with the data at hand. This process of model estimation serves two purposes: first, it serves as a test of whether the range of hypotheses about the interplay between certain events of external science communication and article metrics drawn from past research conforms with a large set of empirical observations; and second, it shall provide a comprehensive view on how the hypothesized interactions compare to each other.

Figure 1 shows the model we start this analysis with. We assume mentions in embargo e-mails and press releases to be exogenous variables, as in almost all cases these events will happen before or very shortly after the promoted article's publication – making it implausible to assume that respective press officers' or editors' decisions could be affected by any of the endogenous variables in our model, which all accumulate later. As a third exogenous variable, we include the publishing journal's prestige, as a large body of research has found this to be a crucial factor for an article's expected citations (see the review by Tahamtan et al., 2016 for an overview of such studies), which therefore cannot be omitted in an endeavor of convincingly modeling the attention articles receive in academic and media spheres. Furthermore, it seems reasonable to assume that also in science journalism there is a common awareness of what the most prestigious journals within a respective covered field of research are – perhaps even of journal impact factors as indicators for such prestige. This makes it likely that journal prestige would be an important variable to explain articles' expected media presence as well (which is also suggested by results from previous case studies such as Dumas-Mallet et al., 2020).

To briefly illustrate the effects assumed in our model with a fictitious example: imagine we have an article that was published in 2018 in the *New England Journal of Medicine* (NEJM), an indubitably highly prestigious journal in its field

(into our model's estimation, this prestige would in this case enter as the median of citations that NEJM-articles received over the years of 2015 to 2017, as our article was published in 2018). Based on past studies (see below), we believe this prestige to have a positive effect on the likelihoods with which journalists will choose said article for their stories, users will post about it on social media, and researchers will cite it in their works. The article's presence on mainstream and social media will likely be higher if the article also received promotion by means of an embargo e-mail or a press release – and the media presence itself will have a positive effect on the likelihood of the article being cited within academic publications as well.

**Figure 1:** Path model with three exogenous variables (Ex1-3), three endogenous variables (En1-3), nine assumed effects between these (a1-9), and one assumed covariance (c1).

The reasoning behind the supposed effects in our model (a1-9) is based on positive associations found in past empirical studies:


Li & Thelwall (2012) and Thoma et al. (2015). Studies on the positive correlation between mentions in news media and future citations (a8) have already been discussed in detail in this article's introduction (Dumas-Mallet et al., 2020; Fanelli, 2013; Kiernan, 2003; Phillips et al., 1991). The (varying) potential of social media-based altmetrics to predict later citations (a9) is proposed by another rich body of studies (see, for instance, Konkiel et al., 2016; Priem et al., 2012; Thelwall & Nevill, 2018).

While cases are imaginable in which high numbers of citations for an article precede it being mentioned in news media or on social media, we for now consider this a less likely case due to findings that indicate that most attention around research articles in social and news media usually happens soon after their publication (Brainard, 2022; Waltman et al., 2021), while the majority of a scientific article's citations typically occur two to seven years later (Schloegl & Gorraiz, 2010). Similarly, a positive direct effect of having a press release on expected subsequent citations is conceivable. However, the findings by Fanelli (2013) indicate that this effect likely is already covered to a large extent by the combined effect of positive associations between press release promotion and mainstream media mentions (a3) and the positive correlation between mentions in news media and future citations (a8). Finally, regarding the presumably complex reciprocal relationship of social media content spawning news content and vice versa – as for example supported by correlations found by Haustein et al. (2015) or Lemke et al. (2022) – we do not assume a unidirectional causal effect, but model the relationship as a covariance (c1) instead.

We use maximum likelihood estimation with robust standard errors and a Satorra-Bentler scaled test statistic (Satorra & Bentler, 2010) for model estimation, as citation and altmetrics data are usually not normally distributed. For the evaluation of the models' global fit, we also consult the (robust) comparative fit index (CFI), root mean squared error of approximation (RMSEA), and standardized root mean squared residual (SRMR) and apply established cutoff criteria proposed by Hu & Bentler (1999). To evaluate models' local fit and identify potential improvements, we calculate modification indices with a cutoff value of 10 (Regorz, 2021). All statistical analyses are performed in R (R Core Team, 2020).

### **Data sampling**

To analyze the relationships between mentions of research articles in external science communication and their performance regarding citations and altmetrics, we start with data obtained from *EurekAlert!*<sup>2</sup> . EurekAlert! is a platform for the distribution of research-related press and news releases that was set up by the American Association for the Advancement of Science in 1996. The platform enables publishers, universities, research institutes, corporations, government agencies, and eligible organizations that engage in scientific research to disseminate press releases to journalists and the public against payment of submission fees. With over 14,000 registered journalists from more than 90 countries, EurekAlert! has become the most prominent multilingual platform of its kind (Vrieze, 2018). Or, as Vrieze (2018) put it: right now, EurekAlert! has become for science press and news releases "what *Google* is for searching and *Amazon* for online shopping."

We focus our analysis on research articles published between 2016 and 2018 to balance the timeliness of the research analyzed in our study with the ability to obtain meaningful citation windows. Thus, the starting point for our dataset is data on 84,194 press releases provided to us by EurekAlert! that were published on the platform between 2016 and 2018. Most of these (79,419, 94.33%) are categorized by EurekAlert! as type "research." In addition, EurekAlert! press releases on new research articles frequently contain a DOI link to the respective article – this enables us to extract 41,937 DOIs. Of these identifiers, 34,055 refer to a valid Web of Science record with publication type "journal," document type of either "article" or "review," and a publication year between 2016 and 2018. These 34,055 DOIs form the starting point for our bibliometric analysis.

As a next step, we enrich this data with information about the presence of research publications in publishers' embargo e-mails to journalists. For this, we rely on data from the Science Media Center Germany (SMC). The SMC is an editorially independent non-profit institution with the mission of supporting journalists in reporting on science-related topics. As one service contributing to this mission, the SMC regularly sends out comments by scientific experts on new research findings that are still under embargo. To be able to provide this service, the SMC aims to monitor as many scholarly journals that send out embargo e-mails as possible. Since 2016, the SMC has accumulated 2,638 ingoing e-mails identified as

 **2** https://www.eurekalert.org/

embargo e-mails from scholarly publishers. Each of these embargo e-mails contains information about one or more upcoming articles from one or more journals belonging to the same publisher.

As we rely on DOIs to track articles' citations and altmetrics, we query the SMC's e-mail archive for embargo e-mails containing references to scientific articles via DOI. This led to 953 articles with Web of Science records of document type "article" or "review," publication type "journal," and publication year 2016, 2017, or 2018 that also have been promoted in an embargo e-mail between 2016 and 2018 with reference to a DOI. Merging these articles with our dataset of 34,055 articles promoted on EurekAlert! enlarges our sample to 34,413 articles that received promotion in an embargo e-mail and/or a press release on EurekAlert!. Table 1 shows to which extent the two regarded types of promotional activities overlap within our dataset. The two events are not statistically independent from each other (Fisher's exact test, two-sided, *p* < .001).


**Table 1:** Contingency table of promotion in embargo e-mails and on EurekAlert! for research articles in our sample.

We also added "control group articles" to our dataset – articles which, to our knowledge, did not receive any promotion in EurekAlert! press releases or publishers' embargo e-mails, but otherwise should have been published under comparable circumstances as our "treatment group articles." The addition of a control group enables us to assess the effects of external promotion via such comparison. To do so, we match every article from the treatment group (i.e., the group of articles that received the "treatment" of getting promoted within an embargo e-mail, an EurekAlert! press release or both) to one random article from Web of Science that was published in the same publication year and with the same ISSN, but which is not part of the treatment or control group yet. We again restrict our matching to the Web of Science document types "article" and "review" to avoid matching research articles with, for instance, editorials, letters to the editor, etc.

For articles from multidisciplinary journals (e.g., *PLOS ONE*, *Nature*, *Science*), this procedure might lead to suboptimal matchings of articles from domains with highly heterogeneous citing or publication behaviors – it would, for instance, be questionable to match a biomedical article from *PLOS ONE* with a sociology-related article from *PLOS ONE.* For articles published in journals classified as multidisciplinary in Web of Science (24.65% of our sample), we therefore apply an additional step, inspired by the matching methodology described in Fraser et al. (2020). To do so, we reclassify these articles and all potential control group articles' disciplines based on the most frequently cited Web of Science subject categories among their references and subsequently use concurrence among these new classifications as an additional matching criterion for articles from multidisciplinary journals.

It should be noted that for a relatively small number of articles from our sample (3.62%), our control group matching procedure does not return a valid match fulfilling the criteria explained above. Thus, a total of 67,581 unique publications serves as our dataset for model estimation.

For bibliometric analyses we use data provided by the *Competence Centre for Bibliometrics* (CCB). The CCB administers databases based on Web of Science, which are updated annually. The bibliographic and citation data used in this study therefore reflects the status of Web of Science from April 2020. The altmetric data used in this study (i.e., articles' numbers of mentions in mainstream media and on Twitter) was obtained via the API of Altmetric.com in November 2021.

### **Results**

Before model estimation, we briefly examine some of the articles' metadata to achieve an understanding of our dataset's composition. In total, 3,419 individual journals are represented within our dataset, the most frequently represented journals being *Nature Communications* (5.38% of all articles), *Scientific Reports* (4.26%), *PNAS* (3.96%), *PLOS ONE* (2.62%), and *Nature* (2.57%). The frequency of the remaining journals follows in a long tail distribution, with most journals (1,450) only represented by a single article within our treatment and control group. Applying traditional Web of Science subject categories, a total of 241 different categories are represented in our data. Table 2 shows the ten most strongly represented journals and Web of Science subject categories from our sample and their respective shares. The outstandingly high share of the category Multidisciplinary Sciences (15.32%) appears to back up what the examination of most commonly represented journals had also shown – namely, that prominent multidisciplinary journals like *Nature*, *PNAS,* or *PLOS ONE* account for a large share of the research that gets featured in press releases and/or embargo e-mails. Also, a look at the most heavily represented categories suggests a particularly substantial representation of research dealing with biomedical subjects.


**Table 2:** Most frequent journals and Web of Science subject categories among our sample.

Four of the six variables in our model are metrically scaled: articles' citation counts, numbers of mentions in tweets, numbers of mentions in mainstream media (MSM mentions), and the median of citations articles within the respective journal received during the past three years (abbreviated as JCM or journal citation median). Table 3 shows descriptive statistics of our sample regarding these variables.

**Table 3:** Descriptive statistics of metric variables within the model.


According to our research aim and method, we estimate a path model based on our publication data and the specification outlined in Figure 1. To account for the considerable differences in their variances, the four metric variables included in our model (citations, tweet mentions, mainstream media mentions, and journal citation median) were all standardized via z-transformation (subtraction of mean and division by standard deviation for each observation) before model estimation.


**Table 4:** Global fit indices for our first model.

**Figure 2:** Final path model with path coefficients. \*\*\* indicate *p* < 0.001.

We evaluate our model's global fit before we consult the estimated model's coefficients. As a result, the *χ2* test for exact model fit is significant (*χ2* = 11.503, df = 2, p = .003). The comparative fit index (CFI), root mean squared error of approximation (RMSEA), and standardized root mean squared residual (SRMR) for our model are shown in Table 4.

Applying cutoff criteria recommended by Hu & Bentler (1999), the CFI > 0.95, RMSEA < 0.06, and SRMR < 0.08 all indicate an already relatively good fit between our hypothesized model and observed data. The significance of the *χ2* test is undesirable, so we are interested in whether further substantial improvements to the model are possible. Using the lavaan package's function modindices, we calculate modification indices to see if and how the change of existing or addition of further effects to our model could increase its fit with the data. The suggested model change associated with the highest expected model improvement is the replacement of the unidirectional effect a8 by a covariance – so the abandonment of the assumption that mainstream media mentions have a mostly unidirectional effect on citations in favor of a model in which no clear causal direction between citations and media mentions is assumed. As such a non-unidirectional relation might make sense from a theoretical standpoint as well, we respecify the model accordingly and again assess its global fit through a *χ2* test (*χ2*= 2.161, df = 2, p = 0.339) and the calculation of fit indices shown in Table 5.


**Table 6:** Final path model's parameter estimates.

Both *χ2* test and fit indices indicate an improvement in global model fit compared to the first model. As another iteration of modification index calculation does not reveal any further model changes that would be linked to substantial expected improvements (applying our cutoff value of 10 for modification indices), we continue with the interpretation of the model's coefficients. Figure 2 shows the resulting model along with its path coefficients and covariances, and Table 6 provides additional statistics. Table 7 shows *R²*-values for the three endogenous variables within the model.

**Table 7:** Endogenous model variables' R²-values.


#### **Discussion**

The obtained fit indices and significances of effects suggest a good fit between the model we hypothesized based on findings from past case studies (Chapman et al., 2007; Dumas-Mallet et al., 2020; Fanelli, 2013; Kiernan, 2003; Konkiel et al., 2016; Lemke, 2020; Lemke et al., 2022; Li & Thelwall, 2012; Phillips et al., 1991; Priem et al., 2012; Stryker, 2002; Tahamtan et al., 2016; Thelwall & Nevill, 2018; Thoma et al., 2015) and the actual publication data at hand. Thus, the first stated purpose of our analysis, which was to test whether our hypotheses about the interactions between events of external science communication and article metrics drawn from past studies conform with a large set of empirical observations, can be considered as fulfilled.

Regarding the effects of exogenous variables on various media mentions, our final model suggests that promotion with embargo e-mails seems to affect articles' expected mainstream media presence to a slightly larger (0.605) but similarly high degree as promotion with a press release (0.568). Furthermore, considering social media presence, the comparatively higher strength of embargo email promotion as a predictor for later mentions becomes even more apparent (0.369, opposed to 0.188 from press releases). As it might seem counterintuitive that embargo e-mails would to a greater extent contribute directly to an article's visibility on various media than the more openly accessible format of a press release, these findings might also suggest that embargo e-mail promotion (as tracked by the Science Media Center Germany) is – compared to press release promotion – reserved for even more elite research publications, which due to innate qualities not represented in our model (e.g., particular originality, societal value, or some form of provocativeness, to just name a few possibilities) will likely attract more media attention on their own. If the selection of articles for embargo email promotion indeed typically follows more rigorous criteria than the selection for press releases, this finding might also be considered a hint towards the validity of the earmark hypothesis as suggested by Phillips et al. (1991). However, further research on publishers', journals', and institutions' criteria behind the selection of research articles for both these forms of promotion would be necessary to solidify this hypothesis.

Our third exogenous variable, journal prestige, proves to be a significant predictor for citations (0.263), mainstream media mentions (0.140), and Twitter mentions alike (0.163) – the direct effect on citations, however, is most substantial. What might be considered a surprising finding are the fairly weak relationships between both mainstream and social media mentions and citations (0.054 and 0.076, respectively). Possibly, controlling for journal prestige (which, as we have seen, is also a strong predictor for mentions in both forms of media) already accounts for most of the citation advantage expected from increased media presence; this interpretation is in contrast to findings by Dumas-Mallet et al. (2020) though, who for their biomedical sample found the expected citations of articles from lower impact journals to benefit particularly strongly from media mentions. Another interesting and perhaps surprising insight concerns the observation that led to our model respecification, namely that a model without the assumed directional effect of mainstream media mentions on citations fits the empirical data better than our initial model, where this effect was present. This finding might hint at a more bidirectional relationship between academic impact and media attention than past case studies suggested, which in line with Phillips et al.'s (1991) publicity hypothesis often focused on how media exposure might increase citations without much consideration of the opposite phenomenon of outstanding research evoking media attention.

Our findings add to existing case studies on associations between media coverage of research and said research's impact by taking a more comprehensive perspective than past studies, which mostly focused on fewer variables, and by analyzing a large sample of articles from a wide variety of journals. Also, to the best of our knowledge, there has neither been a comparative analysis of the effects of embargo e-mails and press releases before, nor an application of path analysis in a large-scale bibliometric analysis like ours. For the interpretation of the results, readers should however keep in mind that our sampling approach started with articles featured on either EurekAlert! or within an embargo e-mail tracked by the SMC Germany and that our study therefore remains a case study whose generalizability might be limited by its sample. Moreover, with its limited number of variables considered, our study can only serve as a starting point for disentangling the complex relationships and effects between the systems of science communication and academic reputation.

Our study comes with some further limitations. First, it is virtually impossible to prove that "control group articles" did not ever receive any kind of external promotion similar to a press release or an embargo e-mail that was not tracked by our data sources. However, as both observed kinds of promotion will most likely be the exception rather than the rule among randomly sampled articles as done in this study, we consider it unlikely that this limitation would have impaired our results significantly.

A second limitation results from our reliance on DOIs to identify references on EurekAlert! and in the SMC's database. While the high amount of DOIs found within the two data sources (which on EurekAlert! has been increasing over the years, see also Chapter 1 in this book) suggests that they are a common way of referencing articles within them, our DOI-based approach means that our data likely underestimates the total shares of articles receiving promotion within press releases or embargo e-mails.

Third, as is the case for many bibliometric and altmetric analyses, results should be interpreted with the data sources used to obtain metrics in this study in mind, as with our reliance on these sources we also inherit some of their limitations; e.g., the limitation of Web of Science to publications indexed by it, or the limited transparency of what Altmetric.com tracks as mainstream media mentions and what it does not (for a recent assessment of Altmetric.com's news mention data see also Fleerackers et al., 2022). The representation of subject fields encountered in this study (see Table 2) suggests that the promotional formats of science press releases and embargo e-mails topically are dominated by natural sciences and, more specifically, biomedical subjects. The extent to which our model also applies to other, less publicly visible fields of research, e.g., the humanities, is an aspect that future research should investigate more deeply. Moreover, the national foci of EurekAlert! (likely towards sources from the United States, see also Bowman & Hassan (2019)) as well as the SMC Germany (towards press materials sent to journalists based in Germany) might have an influence on our results, which should be assessed more precisely in the future.

Overall, our findings support the existence of statistically significant associations between the promotion of research in science PR (i.e., embargo e-mails and press releases) and impact metrics that past case studies had found for individual parts of our model's components and smaller, more restricted samples of scientific articles (Dumas-Mallet et al., 2020; Fanelli, 2013; Kiernan, 2003; Lemke, 2020; Lemke et al., 2022; Phillips et al., 1991). It thus underlines the importance of further, more in-depth research on the selection criteria with which PR officers and science journalists decide on which research to cover (see, for instance, Ba-

denschier & Wormer, 2012; Broer, 2020), as these criteria might diverge substantially from the characteristics that metrics-based indicators are supposed to reflect when used for evaluative purposes. Also, although not at the center of our inquiry, our look at the discipline- and journal-wise composition of our sample of articles promoted in either press releases or embargo e-mails indicates that large shares of them were published in multidisciplinary mega-journals and cover subjects from life sciences, which would confirm observations of media coverage of science made in previous studies (e.g., Elmer et al., 2008; Lemke et al., 2021).

It is important to emphasize that the effects observed in our model cannot readily be assumed to prove causal relationships between articles' external promotion and increased metrics. While past case studies provide convincing arguments for the significance of such causal, publicity-related effects (the results of the case-control study by Phillips et al. (1991), the region-specific differences in effect size found by Fanelli (2013), and the link between numbers of press mentions and citations found by Dumas-Mallet et al. (2020) can all be considered such arguments), other, non-causal phenomena might explain the associations shown by the model as well. Such phenomena are, for instance, backed up by Weingart's (1998) theory of the medialization of science, which would explain an increasing convergence between the criteria with which press offices select publications for promotion and the criteria with which researchers choose research subjects (which therefore could also be expected to experience overall rises in impact metrics like citations). To clarify the precise degree to which such causal and noncausal links explain the associations seen in our model, additional qualitative investigations of the selection mechanisms behind press releases, embargo emails, and different metric events will be necessary (see also Lemke, 2022). What the model derived in our study can provide, however, is a quantification of the potential magnitude to which promotional measures and impact metrics are linked to each other.

### **Conclusions**

We specified a path model of the interplay between two prevalent measures of external science communication, journal prestige, presence in mainstream and on social media, as well as citations, and tested the model against a large set of empirical data from Web of Science, Altmetric.com, EurekAlert!, and SMC Germany. The empirical results confirmed the significance of the effects assumed in the model and signaled substantial associations between the three exogenous variables and articles' expected later impact in both media and academia. In particular, the results highlighted the considerable potential effects of embargo emails and press releases on (social) media attention and of journal prestige on citations.

Future research should work on providing a more detailed picture of the criteria that affect an individual research article's likelihood of receiving certain forms of promotion in external science communication, as well as investigate which article properties (for instance, considering topics, authors, or publication venues) influence respective promotional activities' effectiveness with regard to impact generation. Furthermore, as citation norms and behaviors as well as the ways media discusses research findings can vary considerably between fields, a worthwhile continuation of this analysis could lie in the specification, estimation, and comparison of discipline-specific models of the interdependencies examined in this study.

Finally, it should be noted that we focused our analysis on six variables that we deemed particularly relevant and that were generated via literature research to explain the phenomena we aimed to explore. However, there certainly are more factors that might affect the likelihood of research articles being promoted in external science communication, featured on various media platforms, or cited within other publications, that could, in principle, be included in an analysis like ours. In fact, we would argue that within a context as intricate as the system of science communication, no matter which number of factors a model considers, there will probably always be more one could add. Nevertheless, the incorporation of further variables would be another promising path for future research that aims to build upon this work to take – for instance, including a parametrization of authors' prestige might lead to new valuable insights on the interplay between promotion and received attention of research articles, to name just one example. Linked to this, a potential methodological continuation of this research would be the utilization of structural equation modeling, where even factors as abstract as "pure luck" could be incorporated into the model.

### **Acknowledgments**

We thank Sophie Rotgeri for her crucial support regarding the provision of data on embargo e-mails by Science Media Center Germany and on press releases from EurekAlert!, as well as Nicholas Fraser for his input regarding the control group matching algorithm for articles from multidisciplinary journals. Moreover, we thank the SMC, EurekAlert!, and the Competence Centre for Bibliometrics for providing us with access to their databases. This study is part of the research project MeWiKo funded by the German Federal Ministry of Education and Research (grant number 01PU17018).

### **References**


Berghaeuser, H., & Hoelscher, M. (2020). Reinventing the third mission of higher education in Germany: Political frameworks and universities' reactions. *Tertiary Education and Management, 26*(1), 57-76. https://doi.org/10.1007/s11233-019-09030-3

Bowman, T.D., & Hassan, S.-U. (2019). *Science news and altmetrics: Looking at EurekAlert!* 5. http://altmetrics.org/wp-content/uploads/2019/10/Bowman\_altmetrics19\_paper\_6.pdf

Brainard, J. (2022). Riding the Twitter wave. *Science*. https://doi.org/10.1126/science.abq1541

Broer, I. (2020). Rapid reaction: Ethnographic insights into the Science Media Center and its response to the COVID-19 outbreak. *Journal of Science Communication, 19*(5), A08. https://doi.org/10.22323/2.19050208

Carver, R.B. (2014). Public communication from research institutes: Is it science communication or public relations? *Journal of Science Communication, 13*(3), C01. https://doi.org/10.22323/2.13030301

Chapman, S., Nguyen, T.N., & White, C. (2007). Press-released papers are more downloaded and cited. *Tobacco Control, 16*(1), 71-71. https://doi.org/10.1136/tc.2006.019034


https://doi.org/10.1177/107769900808500410

Fanelli, D. (2013). Any publicity is better than none: Newspaper coverage increases citations, in the UK more than in Italy. *Scientometrics, 95*(3), 1167-1177. https://doi.org/10.1007/s11192-012-0925-0



#### Markus Lehmkuhl, Nikolai Promies, and Melanie Leidecker-Sandmann

## **6. Repercussions of media coverage on science? A critical assessment of a popular thesis**

**Abstract**: The topic of this paper is the relationship between journalism and science. In order to describe a potentially relevant dynamic in this relationship, the German sociologist of science Peter Weingart proposed the term "medialization." It describes phenomena of change within science, such as the oversimplification or exaggeration of research findings, which are associated with an increased need for public attention within science. This concept focuses on the repercussions of journalism on science. Inscribed in the term is the assumption that journalism potentially has great social significance. At the very least, journalism, or the mass media it dominates, is thought to influence processes of change within science. The paper aims to assess the social impact of science reporting in order to plausibilize the significance of the role played by journalism. This is based on recent, partly unpublished empirical findings by a German-French DFG/ANR project, which relate to the ability of journalism to focus public attention on scientific events and actors. The results are essentially negative, in the sense that journalism hardly, or at best only very sporadically, succeeds in focusing public attention on individual scientific events or actors. Based on the journalism's very limited performance in this regard, we consider it implausible that journalism could be as significant a factor as the concept of medialization indicates.

**Keywords:** medialization, science journalism, media coverage of science

## **Introduction: Contours of the relationship between science and the public**

Diagnoses of the relationship between science and the mass media overwhelmingly emphasize the difference between the two. They assume a clear boundary between science and journalism. A scientific sphere that produces reliable knowledge is contrasted with a journalistic one that conveys or is supposed to convey this knowledge to a large audience (Nielsen, 2009). This act of mediation is seen as deficient and finds expression in corresponding metaphors. There is talk of a gulf, fences, barriers, the relationship is like that of oil and water, etc. (Peters, 2013).

There is no lack of examples to confirm this view. Public communication embeds scientific results, expert opinions, interpretations or even methodological practices in social contexts where they sometimes generate an echo that makes scientists shudder because it seems distorted and/or irrational to them. In Germany, thousands take to the streets to prevent experiments on macaques or the sowing of genetically modified potatoes. Researchers, laboratories, fields have to be protected by the police. On the American stock exchange, Monsanto's share prices collapse. It was found that BT corn can harm butterfly larvae (Scheufele, 2013). In Germany, individual scientists, such as the virologist Alexander Kekulé during the Corona crisis, or the historian Daniel Goldhagen, become media stars with "fans," although these scientists have not yet distinguished themselves in research, to say the least (Weingart & Pansegrau, 1999). In the yellow press, a study praising chocolate as a slimming agent makes world news, although it is just flimflam (Bohannon et al., 2015) etc.

The emphasis on a distance between the two spheres, however, highlights a supposedly relevant social problem and thus arouses interest, especially within academia, but also in politics. It might be debatable whether one should go as far as Hartz and Chappell who, in 1997, saw the supposed divide between science and journalism as a "threat to the American future" (Peters, 2013).

Problems are well suited to arouse interest. The more plausible and disturbing they are, the more demand for solutions they create. Barriers have to be overcome; fences have to be torn down. In any case, the competing interpretation, no less plausible, that there is no problem, that public conflicts about science, truth, and rationality are inevitable consequences of science entering the public sphere, does not seem to have found widespread acceptance (Kohring, 2005).

Accordingly, numerous initiatives all over the world have addressed the distance, the gaps and barriers. To the extent that these initiatives referred to journalism, as in Germany from about 1988 onwards, they aimed to overcome the supposed gap with a kind of partnership between science and journalism that would serve to convey science to the public in an accurate and competent way (Lehmkuhl, 2012; Peters et al., 2020). How successful these initiatives have been is difficult to judge. At the very least, in Germany and other European countries, as well as in the USA previously, a relatively small group of journalists has gradually emerged that specializes in science. It shares scientific values (Nielsen, 2009) and has developed professional standards that differ from those of other

journalists, making scientific reputation, for example, a criterion for selecting scientific experts (Lehmkuhl & Leidecker-Sandmann, 2019).

However, the formation of a professionalized group of science journalists and the establishment of improved reporting standards has not changed the dominant, deficit-oriented view of journalism. Objectively, this group may be too small for that. But even systematic empirical analyses, which have never really been able to convincingly substantiate the alarming talk of a large gap or barrier between the two spheres, have done little to change this. The results of scientist surveys, conducted since the seventies, can be cited as an example. While they do not indicate a completely conflict-free relationship, they do not uncover unbridgeable barriers either. All in all, the relationship appears to be fairly harmonious (Peters, 2013).

A theoretically relatively elaborate perspective on the relationship between science and journalism is the concept of the "medialization of science." It assumes a clearly definable boundary between science and journalism. Unlike a "gap" or "barrier," medialization does not fundamentally make a qualitative statement about the relationship. Instead, this concept focuses on the repercussions of journalism on science. Inscribed in the term is the assumption that journalism potentially has great social significance. At the very least, journalism, or the mass media it dominates, is thought to influence processes of change within science. In the context of the medialization thesis it is nevertheless necessary to mention the deficit-oriented view of journalism in order to make plausible what its social and scientific relevance is based on.

As early as 1963, Jürgen Habermas speculated that the repercussions of journalistic reporting could be useful thanks to their wide dissemination: socially and scientifically highly relevant new findings would not disappear into hyper-specialized attention niches from which they would emerge, if at all, only after a considerable delay. Preventing the flow of scientific communication from drying up across specialization boundaries (Habermas, 1969) would only be achieved in some cases by long diversions via readily understandable reconstruction. However, the medialization thesis gains its "charm" primarily from the fact that such feedback is not welcomed as desirable but, on the contrary, is seen as alarming or at least disturbing (Corsi, 2005; Franzen, 2011, 2012; Weingart, 2012). In this respect, this thesis directly follows the dominant perspective in science communication research.

The aim of this article is to stimulate reflection on potential repercussions of journalism on science starting with one of the prerequisites of medialization, the great social significance of journalism. Our focus is on the performance of journalism in focusing public attention on science. This perspective is essential in the

context of the medialization hypothesis because it has so far remained largely unclear whether the social significance of journalism is genuinely as great as the term medialization implies.

To this end, we will proceed as follows: Firstly, we will sketch the theoretical foundations of the concept relatively briefly. Secondly, describing individual cases, we will highlight the difficulties inherent in this concept from the point of view of empirical journalism research. We will then present a selection of findings from a German-French project that has spent the last five years identifying structural characteristics of science reporting. Finally, we will discuss how plausible systemic repercussions are in the light of the empirical findings.

#### **Medialization of Science**

Medialization presupposes a functional differentiation between autonomously operating social systems (Marcinkowski & Steiner, 2010). If we consider theoretical analyses of the public sphere and journalism, which, especially in Germany, have been presented in the debate on Niklas Luhmann's theory of social systems since the beginning of the 1990s, then the public sphere and one of its performance systems, journalism, have developed into a system in a recursive communicative process. Without journalism, modern societies would be incapable of becoming aware of the interdependencies between their parts. Journalism is presented as a system that has addressed this problem professionally. The boundaries of this system are determined by the basic distinction of whether or not an event could attract attention in other social subsystems. This is what journalism specializes in as a performance system in the public sphere. It is the (only) entity that can create and bundle broad attention for relevant topics, i.e., topics that can be connected across subsystems (Kohring, 2004).

This distinction marks the essential difference to science, whose boundaries have been formed by the guiding distinction true/false. As a result, science speaks about events that could be true. All selections made by science are geared to this sense-constituting distinction. The science system is thus specialized in the production of reliable knowledge.

In the concept of medialization – which the German sociologist of science Peter Weingart transferred from political communication research to science research (Weingart, 2001) – two premises are usually conflated:


As a collective term, medialization describes "supra-individual phenomena" that occur in science "as a result of the differentiation of a media system with its own logic as well as the respective given need for public attention" within science (Marcinkowski & Steiner, 2010). With regard to actor theory, medialization thus directs attention to the actions of scientific actors which aim to generate and concentrate public attention by means of mass media and which produce a "supraindividual phenomenon" within science. The term is thus to be understood as a collective term for change within science that is caused by a greater need for attention to science, for which it is dependent on the performance of journalism.

The concept acquires its social relevance when the attention-seeking selection activities of scientific actors (authors, reviewers, editors of scientific journals, scientific organisations, publishers) come into conflict with those aimed at the genesis of true statements (Franzen, 2011). They can lead to exaggeration or promote a choice of topics that is oriented towards current trends. In summary, the term refers to practices that make a scientific result or a project appear in a way that does not correspond to the scientifically agreed requirements of accuracy and/or methodological rigour. These practices are chosen because they are expected to achieve greater publicity, which in turn requires the performance of journalism specialized in generating and concentrating attention. Instead of accurate information, image-building, publicity, and self-promotion take over, which does not promote trust in science but endangers it (Weingart, 2022).

In addition to the various spectacular cases of fraud in social psychology, physics or stem cell research, the recent history of science includes numerous other, less spectacular individual cases that point to such a conflict, such as the alleged discovery of bacteria that can metabolise arsenic instead of phosphorus – a claim that has since been falsified, which was quite obviously due to inappropriate research practices that should actually have been uncovered in the peer review process (Lehmkuhl, 2011). Or take the hype about "Ida," a very well preserved primate fossil from the Messel Pit, over 40 million years old, which was briefly inflated by Norwegian palaeontologist Jörn Hurum in association with three TV stations to become probably the most famous fossil in the world (Nowotny, 2011).

It is therefore fair to say that one cannot deny the plausibility of this concept. At the same time, it proves to be exceptionally challenging when it comes to quantifying empirical social research specialized in journalism and/or science, which strives to embed its findings in a context of medialization. There are two main reasons for this:


(Gilardi et al., 2022). Even if one neglects this aspect, the concept incorporates the problem, which is well known from media effects research, that media influences cannot be convincingly separated empirically from other influencing variables, at least not yet, which makes "hard" evidence of repercussions seem almost unattainable. This is due to the "double role" of journalism which changes the world while describing it (Rosen, 1999). It is, for instance, obvious that mass media play an important role in changing values regarding animal trials. But the question of whether it is the change in values itself, the attention it receives in journalism, or, in return, the feedback from public attention in politics that has an influence on science, is practically unanswerable.

As a result, this concept cannot be falsified by means of empirical social research. At best, it can be made plausible, which is what we will address in the following. In doing so, however, we essentially limit our focus to only one aspect: to the performance of journalism in generating attention for science and research that is external to science. We are therefore concerned with assessing the plausibility of one of the assumptions that is of central importance for medialization, namely the social significance of science journalism.

### **Focusing attention as an achievement of science journalism**

As outlined, we consider the ability of journalism to direct and focus public attention on scientific events as one of the prerequisites for medialization. This is what makes journalism so important in the concept of medialization and justifies the term. If journalism did not regularly focus public interest on scientific events, the medialization thesis would be deprived of one of its essential basic assumptions. Without significant reporting, one could not call the actual media reporting a mediator of feedback between journalism and science. Scientific actors must be able to count on their actions being observed by journalism in order to make possible repercussions plausible. It is plausible, for example, to assume that the actions of professional footballers or top politicians are influenced by the fact that they are under constant media observation. Every gesture, every statement can become the subject of reporting. The players observe this and adjust to it (Meyen, 2014). In principle, it is also plausible that the media presence of certain actors influences the actions of other actors who do not themselves expect media observation. Amateur footballers, for example, might copy jubilant poses that they know from the media. However, such effects require that journalism focuses attention on such practices. But how plausible is this assumption even for top actors in academia?

To address this question, we argue that it is not sufficient to merely look at the extent of scientific references in reporting – the increase in references to science in mass media in the last decades, which we can assume (Bauer et al., 2006; Bucchi & Mazzolini, 2003; Elmer et al., 2008) does not convincingly support the assumption that journalism is permanently observing science or parts of it. An increase in scientific references can, for example, come about because political actors cite scientific references more frequently or involve scientific experts more often in political decision-making processes (Weingart & Lentsch, 2008). In this case, references to scientific content and scientific actors do increase in the reporting. However, they are not the result of a change in journalistic observation of science but are of a secondary nature. They are a consequence of changes in the political system. In this respect, the increase in scientific references would not be a change that points to an increase in the importance of science for journalism. Instead, they primarily refer to changes in politics – what one could perhaps call the "epistemisation of politics" (Bogner, 2021) – the consequence of which might be, among other things, a greater presence of scientific experts in reporting. If such a dynamic led to repercussions within science, which would be conceivable, then one would have to call this the politicization of science rather than medialization.

In order to be able to make feedback effects more plausible, we believe one must look more closely at the journalistic achievement of directing and concentrating attention on units that are particularly relevant for feedback processes and that are amenable to empirical analysis. It is – we argue – not only the number of references that matters, but also their distribution.

In the context of science, these relevant units include, in particular, study results and scientific experts or, more generally, scientific actors. These units are pragmatically relevant because they are relatively easy to grasp empirically. But they are relevant above all because observable changes in these units (studies exaggerate their relevance, statements oversimplify a fact, etc.) can be linked to actual media attention or more generally to "media logics."

In the journalistic context, results or expert statements become prompts for reporting, with feedback becoming more likely if the journalistic selections are suitable for focusing public attention on very specific results and scientific actors. This can be plausibly inferred when individual results or actors are not only

picked up on by individual editorial offices, but when the journalistic system – understood here as a unit consisting of a multitude of individual journalists – pays attention to certain prompts. Especially when individual journalists select prompts, topics or experts congruently, public attention is generated to an extent that makes feedback from this publicity more likely.

#### **. Study results as units of journalistic observation**

In the next section, we want to look in rather more detail at study results that have sought to estimate the degree to which journalism is able to focus public attention on the results of certain scientific studies. To this end, we conducted several sub-studies:


The four studies aimed at finding structural similarities in journalistic selection processes that are expressed in very small as well as in very large samples. This research design was inspired by the question of whether something that the mathematician Benoit Mandelbrot called "fractals" also develops in social systems. We wanted to know whether the journalism system could be described in a similar way to a tree, for example, whose trunk branches out into several large main branches according to a certain pattern. The main branches in turn branch out according to the same pattern, then the branches themselves. This continues into the small leaf-bearing tips (Brockmann, 2021).

The results indicate that this seems possible. We find astonishing structural similarities in both the very small and the very large samples. Journalistic selection seems to follow clear mathematical rules that are so common that the term "power law" has been used to describe them (Newman, 2005). The actually measured congruence-frequency distribution can be described almost exactly in all the studies mentioned so far with a basically simple formula that relates the congruence, i.e., the number of media titles that select an event (K), and their frequency: H=1/Kalpha. The parameter alpha of the right-skewed distribution was a good three in both the small and the big samples (Lehmkuhl & Promies, 2021; Lehmkuhl & Promies, 2020).

Apart from these structural similarities, our results show that the wide dissemination of individual scientific results is rare to very rare. Considering the small national sample of print and online titles in detail, we come to the following generalized conclusion: out of 100 scientific events that are picked up by German print and online journalism in an ordinary week, about 90 are taken up by one title only, whether by online titles or print titles. That a scientific event is taken up by more than half of all media titles in these small samples does not occur in an ordinary week. The selection of individual events is thus much less congruent than in almost all other journalistic departments. In politics, economics, culture, and miscellaneous, 70-75 of 100 events in a week are covered exclusively by one title, two-three by all.

These results concur almost completely with the findings of earlier studies, one of which, a good ten years ago, specifically examined the selection made by the science sections of nine different German media titles, including radio and online titles (Wilhelm, 2008). Other analyses going back 20 years, specifically of political reporting, also coincide with our findings (Rössler, 2002, 2003).

With regard to science journalism in Germany, we find firstly no evidence that the ability of the journalistic system to focus attention on individual scientific events has changed, at least not in the last ten years. And secondly, we must state that the ability of the journalistic system in Germany to focus public attention on scientific events is comparatively low, certainly significantly lower than in other fields. Although numerous events are picked up, the selection is extraordinarily diverse. There is singing, one might say, but no choir.

We successfully reproduced this structural feature of the journalistic selection of individual scientific events in two further sub-studies with very large samples (Kohler et al., 2020; Lehmkuhl & Promies, 2020). We used the msm-score from Altmetric.com as an indicator. This score ostensibly measures the distribution of individual scientific studies in about 2,000 journalistically dominated online media. In validation experiments, however, we came to the conclusion

that Altmetric's msm scores smaller than 50 do not reliably indicate that journalism has selected a result. Only above this value can one assume that isolated journalistically dominated online titles on national media markets (and not mere aggregators of press releases such as EurekAlert!) have actually picked up a study result. From a value of approximately 100, a slightly congruent selection can be assumed. Above this value, it is likely that more than one media title on a national media market has taken up the result.1 For all of the approximately eight million studies that appeared in Scopus between 2014 and 2018, we estimated how often they were picked up by journalist-dominated online media. If we only look at the results that were picked up congruently, then journalism generated public attention for about one to two studies out of 10,000 in the period mentioned.2 

Based on these findings, we assume that the structure of journalistic selection processes found in the small, national samples is not a German peculiarity, but also prevails in other national media markets. Only a tiny part of the scientific study output is even mentioned in journalistically dominated dissemination media. And of this tiny part, journalism again only focuses public attention through congruent selection on a small proportion: approximately one tenth of the studies selected. A strong focus of attention by means of a highly congruent selection by the journalism system does probably occur but is rare to very rare.

We have argued that bundling attention to study results is conceivable as one mechanism for making feedback plausible. However, in relation to our findings, we have to state that the journalistic system only rarely achieves this. On the basis

 **1** Our validation was based on 1,601 scientific articles that were published in the journals *Nature*  and *Science* between January and October 2017 (extracted from Scopus database). We collected the msm-scores of these studies and assigned them to 11 groups, namely scores of 1–9; 10–19; 20–29;…; ≥ 100. Subsequently, we conducted a manual search of randomly selected sets of five research articles per group (N = 55) in the full-text press database Nexis to determine from which score we could infer journalistic coverage on three large national media markets (the United Kingdom, the United States, and Germany). The amount of press coverage was classified as "noteworthy" if more than 15 articles on at least one of the five studies per category had appeared in these media markets. Our results showed that only a score ≥ 50 indicates that single media titles pick up a scientific paper. From a score of ≥ 100 it can be assumed that a result has been taken up congruently by a larger number of media titles in different countries (USA, UK, and Germany), indicating a broad international dissemination. Additionally, we researched studies with no Altmetric.com scores, since it is known that many studies are not captured by Altmetric.com. We therefore used a random sample of 100 results not captured by Altmetric.com to see if media titles in the three media markets USA, UK, and Germany reported on any of these studies. This was not the case.

**<sup>2</sup>** It should be emphasized that this is an estimate.

of these analyses, we find no evidence for the assumption that feedback on science can somehow be influenced by broad, journalistically mediated public attention or extends beyond individual cases.

#### **. Scientific actors as units of journalistic observation**

Focusing public attention on study results is only one possible mechanism that can plausibilize feedback. Another is public attention on scientific actors who serve as sources for journalism. Again, we argue that it is not only the extent of referencing scientific sources that is relevant for plausibilizing feedback on journalistic selection, but, additionally, the distribution of these references. With regard to the plausibility of feedback relating to actors, it is also important how high the share of those sources is in the mass media, if they are present at all. However, it also depends on how this presence is distributed. Feedback that can generate "supra-individual phenomena" is plausible where actors can count on their communicative actions being observed by journalism.

This point can be illustrated with an example from political communication. Plausibility has been convincingly demonstrated in the German parliament where opposition politicians increasingly submit so-called "minor requests" to the government because they have observed that it helps them increase their chances of attracting media attention. This has led to the "supra-individual phenomenon" that the number of minor-requests has increased over time. The observation that certain communicative acts are successful thus sets a dynamic in motion that can be called the medialization of politics. Essentially, for this dynamic to gain momentum, the preceding observation about the success of a practice, i.e., actual media coverage, is a precondition (Jandura, 2007).

What can be said about public attention for scientific actors? First of all, we can basically assume that scientific experts are quoted more frequently by journalists today than two, three or four decades ago, although reliable studies on this are rare and do not consistently confirm a growing trend (Huber, 2014). One study from Denmark shows a considerable increase in journalistic references to scientific experts in three Danish print titles by a factor of three for the period between 1961 and 2001, with the highest growth between 1991 and 2001. Social scientists and humanities scholars accounted for the lion's share of this growth, while references to hard scientists increased only very moderately in this study (Albæk et al., 2003). We ourselves surveyed recently all references to individual actors in 1,855 articles, each of which appeared in an artificial week in 2000 or 2019 in six German media titles. In both periods, about 2,600 different actors are cited by journalists. In 2000, eight percent of these references were to scientific

experts; twenty years later, it was 11 percent. The enormous growth in references to scientists in Denmark in the 1990s does not seem to have continued after 2000, at least not in Germany. Nevertheless, the share of approximately 11 percent in the most recent period is noteworthy if one compares it with the share of references to members of the government, which amounts to 16 percent.

As described, the plausibility of a feedback mechanism also depends on how these references are distributed. To the best of our knowledge, there is only one reliable finding from Switzerland on this topic in relation to individual academic experts. It surveyed the media presence of all 6,000 professors in Switzerland in national and international media. Almost 70 percent of these professors did not appear by name in the course of 2016. The journalistic references in the predominantly Swiss media titles were allotted to 1,877 professors, the majority of whom appeared only once within a year, i.e., once within a year in one of dozens of Swiss media titles. The bulk of attention went to a small group of 188 professors, or three percent of the total, who accounted for 50 percent of all references (Rauchfleisch & Schäfer, 2018).

We ourselves surveyed the number of sources on which international journalism based its selection of study results in the period between 2014 and 2018, and how the selection of sources was distributed. We therefore examined from which journals the study results originated that international journalism selected for reporting, once for all study results (Lehmkuhl & Promies, 2020) and once for studies that can be attributed to the relatively popular neurosciences (Kohler et al., 2020).

The results are quite astonishing when one compares the number of times individual journals are mentioned as sources in journalism with those of individual professors from Switzerland. Fifty-five percent of all 1,236 journals that were mentioned in mass media were cited once in the four-year study period, which is approximately three percent of all journals listed in Scopus. The top decile of sources, i.e., the 120 journals with the most mentions, accounted for about twothirds of all references. Both distributions are very similar. According to this estimate, the probability that a journal with *n* references in journalism will be referenced a second time regularly decreases by *x-2* (Lehmkuhl & Promies, 2020). This is approximately the same as for Swiss professors.

Like the study results, the journalistic system focuses public attention only on a small section (individual scientists in a national context) or tiny part (scientific publishers) of the actors.

However, the focus on individual actors as sources of information and expertise is strong. For this very small group of individual actors, repercussions from

public attention are at least plausible even though empirical evidence is rare: individual scientific actors might be more frequently requested for lectures, might have greater chances of becoming members of political advisory bodies, while their study results receive greater attention in the scientific community, etc. (Dumas-Mallet et al., 2020). Even more significant, however, is that their strong public presence cannot be explained without assuming that the actions of journalistic actors and these individual scientific actors have been coordinated. But given the low frequency, it is doubtful whether such feedback is systemically relevant.

Feedback is also plausible in the case of the journals on which journalism concentrates, especially if we also take into account the fact that the journalistic focus on a small number of individual journals, above all Science and Nature, but also Lancet, JAMA or PNAS, has prevailed for at least three decades (Lehmkuhl & Promies, 2020). Such a sustained journalistic focus on these sources cannot be plausibly explained without assuming that the editorial selection in these journals is oriented towards news factors, even if the editors may deny this (Franzen, 2011). Despite this observation, in our view no "systemic" feedback can be plausibly explained, because there are only a handful of scientific dissemination media whose study results are picked up by journalism to any appreciable extent.

#### **. Study results and scientific actors in the context of science-related public debates**

Up to now, we have tried to assess how journalism focuses public attention on individual study results and individual actors in a relatively decontextualized way. In doing so, we have so far ignored the fact that one of journalism's very important achievements is to draw attention to issues and to bundle them. Journalism regularly assigns study results and statements by individual actors to overarching contexts of meaning, which we can call topics (e.g., antibiotic resistance, cancer) or topic groups (e.g., biotechnology) (Kepplinger, 2011), although the boundaries between these contexts can be difficult to determine in individual cases.

In several sub-studies over recent years, we have examined, among other things, differences in journalistic selectivity relevant in this context, namely whether the arbitrariness of the selection of individual study results or of scientific actors as sources for expert opinions changes when public attention focuses on science-related topics, such as the use of glyphosate in agriculture, the role of nitrogen oxides in air pollution control or COVID-19 (see also Chapters 2 and 3 in this book). We have done research on eight topics and four groups of topics,

which, however, essentially only relate to Germany (Kohler et al., 2020; Lehmkuhl & Leidecker-Sandmann, 2019; Leidecker-Sandmann, Attar, et al., 2022; Leidecker-Sandmann, Promies, et al., 2022).

The relatively pronounced concentration of journalism on just a tiny section of journals as sources says little about whether other journals come into play depending on the topic, so that feedback on a wider circle of journals could be plausible under certain circumstances. Surprisingly, however, this is not the case. Essentially, the journalism in all nine individual topics we investigated always bases the bulk of its study selection on the same journals. If we again select the upper decile of influential journals for the overarching description, then almost two-thirds (62%) of all references in the individual topics also occur in this small group of journals, above all Science, Nature, Lancet, and NEJM. In other words, regardless of whether the subject is neuroscience, infections (antibiotic resistance, Ebola, influenza) or environmental topics (dioxin, glyphosate, nitrogen oxides), about COVID-19 or about marijuana, the study results that journalism selects for these topics overwhelmingly derive from these journals. Differences between the individual topics only arise in the composition of the journals from which occasional study results are taken. Here there are differences between the individual topics. Accordingly, we cannot find any indication in this research approach regarding the journalistic focus on individual journals that could plausibilize feedback that extends beyond the tiny circle of the journals mentioned.

However, in terms of topics, the dominance of this small group of journals is so great that it is plausible to assume that other journals will follow suit. If we simplify and assume that in principle every journal has an interest in generating attention in journalism, then the most plausible feedback mechanism is to favor topics on which journalism has concentrated attention. This need not be limited to the top journals.

We explored this question in another study. We investigated whether more thematically similar studies are published by journals after a single study has been congruently selected by journalism. To do this, we used the approximately 1,000 study results that received a lot of media attention in the period between 2014 and 2018. We assumed that the congruently selected study results tended to be the ones that referred to a topic more widely discussed in society. In fact, the number of publications of thematically similar studies increases slightly when a study has achieved broad attention in journalism. This is not restricted to the top journals. The effect also occurs in journals that are not favored by journalism (Leidecker-Sandmann et al., 2023). In other words, in those rare cases in which it

focuses attention on science-related topics, there is a kind of correspondence between the popularity of a certain topic, indicated by congruent selection of journalism, and the topic selection in journals.

We also examined the distribution of references to individual scientific experts in the topic-related studies. Here we find a notable difference compared to the general referencing structure. In the science-related debates, there is generally only a very limited focus on a few, particularly visible, scientific experts. However, the exception is the COVID-19 debate, where this does occur (Leidecker-Sandmann et al., 2022; Eisenegger et al., 2020). Here, there is a greater concentration on a few individual scientific actors, although the COVID-19 debate, like all other topics, is fundamentally characterized by the fact that most of the various scientific experts only have their say at best sporadically. It is the only topic we are aware of so far, which makes individual scientists "stars."

For three groups of topics (biotechnology, neuroscience, climate change) we have tried to estimate the temporal stability of the dominant referencing pattern. We compared the distribution of references to individual scientific actors in the reporting of six German media titles in 2000 with that in 2019. There are hardly any significant differences. In 2019, references to scientific experts are even slightly more sporadic than 20 years earlier. In both periods, approximately 80- 85 percent of all scientists are only cited once by journalists in the six titles studied. Visible scientists do exist in each of the topic groups, but these are individual cases (Promies et. al., [in preparation]).

### **Discussion: On the plausibility of the medialization thesis**

We find very few features in the actual media coverage of science that would serve to plausibilize a significant role played by science journalism. As a rule, journalism hardly focuses public attention on scientific results or scientific actors. It usually does not achieve what is attributed to it in the context of the medialization thesis. This does not mean that the media presence of science does not achieve selective effects. Media attention on individual scientific studies can increase scientific attention (e.g., Dumas-Mallet et al., 2020; see also Chapters 4 and 5 within this book), the public prominence of topics can selectively influence the selectivity of journals (Leidecker-Sandmann et al., 2023). In our opinion, however, such repercussions cannot justify the term "medialization" because this term seeks to describe systemic effects that are not plausible, at least not on the basis of the actual media presence of relevant scientific units.

The low degree of public focus on study results can be explained, among other things (Lehmkuhl & Promies, 2020), by the fact that, unlike politics, business or culture, science is less likely to produce individual events whose news value exceeds the critical threshold at which the entire system of journalism, or at least significant parts of this system, react by reporting. Secondly, it can be explained by the fact that science produces a comparatively larger number of events with similar news value, so that science journalism finds it more difficult to make congruent selections than political, economic or cultural journalism.

The fact that there is seldom a focus on individual scientific actors can be explained, among other things, by the fact that individual experts are generally experts on an extremely small area of world events, which means the journalistic focus is limited to a very small circle of actors. In addition, science has so far hardly differentiated prominent spokesperson roles in the sense that an actor would be legitimized to speak for other scientists or other scientific organisations.

In the context of the medialization thesis, the findings described above on the role of a small group of scientific journals deserve special attention. Our findings suggest that this very small number of journals function *de facto* as agencies for publicly relevant scientific studies. This is indicated by the constancy of their topic-independent dominance as sources of international science journalism, which cannot be plausibilized without the assumption of reciprocal co-orientation processes. We can only hint at the theoretical implications of this finding here. But given the extensive editorial selection practised by these journals even before reviews, it seems plausible that selection processes within these journals rather than the ones in journalism can mediate feedback processes (Franzen, 2011).

Taken together, we find very little empirical substance for the theoretical optics of the medialization thesis when considering feedback from real media coverage. But to stress this point again: these findings cannot falsify medialization, since real media coverage is just one mechanism to justify the term medialization. Another is the so-called "actor fiction" regarding the importance of media presence that seems to be widespread within science, especially within the PR-departments of scientific organisations (Marcinkowski et al., 2014; Marcinkowski & Steiner, 2010). Such a fiction can plausibilize, among other things, a notable increase in the PR efforts of science organizations (Autzen, 2014; Serong et al., 2017; Vogler & Schäfer, 2020), which is cause for concern. With Weingart (2022) and many others, we consider it fundamentally plausible that a bare belief in the importance of media presence heralded by the New Public Management is filtering

through to the core of scholarly communication (Lehmkuhl, 2019). However, we doubt that these developments should be named "medialization" since they correlate insufficiently with actual media coverage. The term blames mass media for something that cannot be sufficiently related to what the mass media actually do.

### **References**


Corsi, G. (2005). Medienkonflikt in der modernen Wissenschaft? *Soziale Systeme, 11*(1), 176-188.


Gonon, F., Bezard, E., & Boraud, T. (2011). Misrepresentation of neuroscience data might give rise to misleading conclusions in the media: The case of attention deficit hyperactivity disorder. *Plos One, 6*(1), e14618. https://doi.org/10.1371/journal.pone.0014618

Gonon, F., Konsman, J.-P., Cohen, D., & Boraud, T. (2012). Why most biomedical findings echoed by newspapers turn out to be false: The case of attention deficit hyperactivity disorder. *Plos One, 7*(9), e44275. https://doi.org/10.1371/journal.pone.0044275

Habermas, J. (1969). *Technik und Wissenschaft als "Ideologie"*. Suhrkamp.

Hanebeck, J. et al. (2021). *Die Kongruenz der journalistischen Anlassauswahl: Eine vergleichende Untersuchung zwischen Journalismus und Wissenschaftsjournalismus in den Online- und Offline-Medien*. (unveröffentlichte Studienarbeit) Karlsruhe. KIT.

Huber, B. (2014). *Öffentliche Experten: Über die Medienpräsenz von Fachleuten*. Springer Fachmedien Wiesbaden. http://dx.doi.org/10.1007/978-3-658-05405-2


Kohring, M. (2004). Journalismus als soziales System: Grundlagen einer systemtheoretischen Journalismustheorie. In M. Löffelholz (Ed.), *Theorien des Journalismus: Ein diskursives Handbuch* (2nd ed., pp. 185-200). Springer Fachmedien.

Kohring, M. (2005). *Wissenschaftsjournalismus: Forschungsüberblick und Theorieentwurf*. UVK-Verl.- Ges.

Lehmkuhl, M. (2011). Getrennte öffentliche Sphären: Die offline Medien berichten über das mutmaßlich Arsen fressende Bakterium so, als gäbe es das Internet nicht. *WPK-Quarterly. Magazin Der WPK - Die Wissenschaftsjournalisten, 9*(1), 4-7.

Lehmkuhl, M. (2012). The recent public understanding of science movement in Germany. In B. Schiele, M. Claessens, & S. Shi (Eds.), *Science communication in the world* (pp. 125-138). Springer Netherlands. https://doi.org/10.1007/978-94-007-4279-6\_8

Lehmkuhl, M. (2019). Journalismus als Adressat von Hochschulkommunikation. In B. Fähnrich, J. Metag, S. Post, & M.S. Schäfer (Eds.), *Forschungsfeld Hochschulkommunikation* (pp. 299-318). Springer VS. https://doi.org/10.1007/978-3-658-22409-7\_14

Lehmkuhl, M., & Leidecker-Sandmann, M. (2019). "Visible scientists revisited": Zum Zusammenhang von wissenschaftlicher Reputation und der Präsenz wissenschaftlicher Experten in der Medienberichterstattung über Infektionskrankheiten. *Publizistik, 64*(4), 479-502. https://doi.org/10.1007/s11616-019-00530-1


Rauchfleisch, A., & Schäfer, M.S. (2018). *Welche Forschenden erscheinen in den Medien? Befunde aus der Schweiz.* wissenschaftskommunikation.de. https://www.wissenschaftskommunikation.de/welche-forschenden-erscheinen-in-den- medien-befunde-aus-der-schweiz-21015/

Rosen, J. (1999). *What are journalists for?* Yale Univ. Press.


#### **Name Index**

Afshar, A. S. 138 Ahamdzadeh, K. 103, 138 Ahmed, R. 49, 76 Aizaki, H. 92, 97, 99 Akhmanova, A. 76 Aksnes, D. W. 118, 136 Albæk, E. 150, 156 Aldama, A. 75 Allan, S. 53, 75 Allard, S. 103 Allen, D. S. 57, 75 Allen, K. 50 Allison, P. D. 97, 99 AlMegren, H. 50 Alperin, J. P. XV, 49, 76-77, 136 Alshihri, S. 50 Altmeppen, K. D. 33, 47 Amin, M. 100 Analatos, A. A. 102 Anderson, P. S. 80, 96, 99 Ardern, C. 50 Arnold, K. 158 Ashwell, D. J 30, 47 Attar, P. 153, 158 Autzen, C. 2, 4-5, 12, 25, 30, 47, 155-156 Backhaus, K. 81-82, 100 Badenschier, F. XIII-XIV, 34, 36, 38, 47, 134, 136, 156 Bahler, C. D. 103 Baker, S 100 Balcom, L. 50 Banerjee, I. 50 Barrett, J. C. 101 Barros, T. 50 Bartsch, A. XVI, 51 Bates, B. J. 103 Bauer, M. W. VIII, XIV, 45, 48, 146, 156 Beck, J. 75 Bednarczyk, B. XVI, 26, 102, 137 Bednarek, M. 36, 48 Behrens, T. E. 76 Bentler, P. M. 124, 130, 137-138

Berg, H. VIII, XV Berger, M. 50 Berghaeuser, H. 120, 136 Beyreis, M. 137 Bhandari, M. 87, 100, 104-105 Bicchieri, C. 54, 75 Biele, G. 77 Birch, B. 103 Birkinshaw, J. 101 Bischoff, J. 76 Björk, B. C. 39, 52 Bloor, M. 85, 100 Blum, R. 159 Boetto, E. 75 Bogner, A. 146, 156 Bohannon, J. 140, 156 Boland, J. J. 50 Bonfadelli, H. XVI, 51 Boraud, T. XV, 25, 76, 100, 136, 156-157 Borgman, C. L. VIII, XV Bornmann, L. VIII, XVI, 81, 84, 86, 103, 106-107, 112, 121, 138 Bowman, T. D. 4, 8, 15, 19, 25, 80, 100- 101, 133, 136 Boyadjieva, P. 156 Bradley, F. 49 Brainard, J. 124, 136 Brede, M. XI, 26, 79, 137 Breuer, T. 80, 98, 100 Bridges, J. F. P. 102 Brierley, L. XV, 49, 76 Brockmann, D. 147, 156 Broer, I. VII, XI, XIV-XV, 29-32, 47-48, 81, 96, 99-100, 134, 136 Broks, P. 50, 158 Brondi, S. 75 Brossard, D. 137 Brüggemann, M. 40, 48 Brumberg, J. 138 Bryson, G. L. 50 Bucchi, M. XV, 30, 48-49, 146, 156 Bucher, H.-J. XVI

https://doi.org/10.1515/9783110776546-007

Burge, P. 102

Burns, T. W. 2, 25, 79, 100 Buschow, C. 29, 32, 46, 48, 52 Busse, J. W. 100-101 Caldwell, C. 103 Callaham, M. L. 87, 100, 104 Cameron, R. 102 Caple, H. 36, 48 Carlson, M. 30, 48 Carson, R. T. 102 Carullo, G. 75 Carver, R. B. 119, 136 Casas, A. 75 Castelfranchi, C. VII, XV Castillo-Valdivieso, P. A. 78 Chan, T. M. 138 Chapman, S. 119, 123, 131, 136 Chi, P.-S. 26 Chiarelli, A. 54-56, 76 Christakis, N. A. 97, 99 Christensen, W. F. 99 Christian, L. 103 Christiansen, P. M. 156 Ciro, J. B. 50 Claessens, M. 157 Classen, C. 158 Clauser, C. 102 Cloître, M. XV Coates, J. A. XV, 49, 76 Cobb, M. 55, 76 Cobey, K. D. 50 Cohen, D. 157 Colavizza, G. 54, 76 Collings, A. 138 Cook, T. E. 40, 48 Corley, E. A. 137 Corman, V. M. 77 Corsi, G. 141, 156 Costas, R. X, 1, 5-6, 22-23, 25, 27, 76, 101, 137 Cottle, S. 33, 48 Crabtree, C 82, 100 Craig, G. 53, 57, 76 Craig, I. D. 87, 100, 104 Cronin, B. 81, 100 Cugusi, L. 50 Cukier, S. 50

Dahlgren, K 100 Dahm, P. 103 Dascal, M. XV-XVI, 49 Davidson, L. E. 99 Delgado López-Cózar, E. 26 Dernbach, B. 80, 100 Deshpande, I. 75 Devereaux, P. J. 100 Dey, G. XV, 49, 76 Dalen, H. P. van 98-99, 101 Dijk, T. A. van 56-57, 76 Dilworth, S. 50 Dolgin, E. 54, 76 Domingo, D. 33, 48 Donaldson, M. R. 50 Donnalley, L. 102 Donsbach, W. 36, 44, 48-49, 159 Dozier, D. M. 43, 49 Driehaus, A. 156 Drosten, C. 66, 71, 75, 77 Dudek, J. 23, 25 Duffy, A. 30, 33, 52 Dumas-Mallet, E. VIII, XV, 3, 25, 58, 76, 80, 100, 119, 122, 124, 131-134, 136, 144, 152, 154, 156 Dunn, L. 101 Dunwoody, S. IX-XV, 30, 44, 49, 53-54, 76 Eck, N. J. van 26, 76 Edelmann, A. 77 Edwards, T. 50 Egger, M. 50 Einhorn, T. A. 100 Eisen, M. B. 76 Eisenegger, M. 154, 156 Ekström, M. 29-30, 33, 36, 44-45, 49, 52 Eldridge, S. A. 50 Elkana, Y. 158 Elmer, C. 134, 136, 146, 156 Enkhbayar, A. 136 Entman, R. M. 57, 76 Erichson, B. 100 Errington, T. M. 77 Ettema, J. S. 29, 44-45, 49 Evans-Pickett, A 99

```
Fagan, K. XV
```
Fähnrich, B. XVI, 51, 157-158 Fahy, D. IX-XV Fanelli, D. VIII, XV, 2-3, 25, 80, 101, 119, 123-124, 131, 133-134, 136 Fang, Z. 138 Fantini, M. P. 75 Farshad, M. 87, 101, 105 Fatas, E. 75 Fedorov, V. V. 89, 101 Ferguson, C. A. 75 Fero, M. 75 Ferrucci, P. 50 Figg, W. D. 87, 101, 104 Figuerola, C. G. 25 Fishman, M. 33-34, 49 Fleck 57 Fleck, L. VII-VIII, XV, 45, 49, 76 Fleerackers, A. XIII, XV, 35, 47, 49, 54, 58, 62, 64, 73, 76-77, 133, 136 Flynn, T. N. 102 Fox, F. 29, 31, 49 Franceschi, D. 102 Franck, G. 98 Franks, S. XV Franzen, M. VII, XIII-XVI, 2-4, 25-26, 30, 34, 46-47, 49, 136, 141, 143, 152, 155- 157, 159 Fraser, N. IX, XV-XVI, 35, 49, 52, 54, 76, 127, 135-136 Friedrichsmeier, A. 30, 49, 158 Fröhlich, R. XVI, 51 Frost, M. 99 Funk, K. 75 Fürst, S. 158 Fyie, K. 102 Gabriel, M. 50 Gajevic, S. 31-32, 52 Galtung, J. 98, 101 Gamson, W. A. 45, 49 Gao, Y. 50 Garenne, A. XV, 25, 100, 136, 156 Garfield, E. 81, 101 Garvey, W. D. 2, 25 Gascoigne, T. 158 Gerber, A. 30, 50 Gerber, C. 101

Gessler, T. 157 Gibbons, M. 51 Gieryn, T. F. 45, 50 Gilardi, F. 145, 157 Glänzel, W. 26 Glaser, B. 62 Glasser, T. L. 29, 44-45, 49 Gloning, T. VII, XV-XVI, 49 Golinelli, D. 75 Gonon, F. XV, 25, 76, 100, 136, 144, 156- 157 Gonzalez-Cortes, S. 50 Gordon, T. J. 84-85, 101 Görke, A. 30, 50 Gornev, G. 156 Gorraiz, J. 124, 138 Graham, I. D. 50 Grant, J. 102 Grant, R. M. 34, 50 Gray, H. M. 99 Greaves, S. 138 Groth, P. 137 Groves, T. 2, 25 Grudniewicz, A. 39, 50 Guenther, L. XV, 31, 46, 50, 58, 76, 78 Gun'ko, Y. K. 50 Guran, P. 75 Guthrie, S. 102 Habermas, J. 141, 157 Hadfield, J. G. 99 Hanebeck, J. 147, 157 Hanitzsch, T. XVI, 51-52 Hansen, A. 48 Hanson, B. 75 Harcup, T. 36, 50 Harrison, M. 75 Hasebrink, U. VII, XV, 30, 48 Hassan, S. U. 4, 8, 15, 19, 25, 80, 100, 133, 136 Hauber, B. 102 Haustein, S. 77, 82, 101, 124, 137 Heeffer, S. 26 Heidbrink, H. 52 Heimstädt, M. 54, 56, 76 Heinrichs, A. 138 Hemminger, B. M. 137

Henninger, F. 88, 101 Hepp, A. 30, 50 Héritier, A. 51 Hermida, A. 34, 50, 120, 137 Hettwer, H. 32, 50, 159 Heyl, A. IX, XV Hicks, D. 99, 101 Hilbig, B. E. 101 Hilgartner, S. VIII, XVI Hobert, A. XVI, 52 Hodgkinson, M. 50 Hoelscher, M. 120, 136 Hofmann, J. 77 Holbein, J. B. 100 Hollingshead, T. 99 Homm, P. 156 Horbach, S. P. J. M. 54, 76 Htoo, T. H. H. 96, 101 Hu, L. 124, 130, 137 Huber, B. 150, 157 Hurley, R. J. 77 Hurst, P. 138 Ide-Smith, M. 75 Imhof, K. 159 Ioannidis, J. P. A. 102 Iravani, E. XVI, 52 Isohanni, M. 102 Jackson, D. 51 Jahn, N. XVI, 52 Jandura, O. 150, 157, 159 Jarren, O. 159 Jie, X. 37, 50 Jitsuzumi, T. 101 Johnson, R. 76 Jones, D. 10, 26 Jones, J. B. 99 Jones, T. C. 71, 77 Joubert, M. IX, XV, XVII, 31, 46, 50 Kanter, E. J. XVI, 26, 102, 137 Karlsson, M. 53, 57, 77 Kaufman, J. 103 Kehoe, D. K. 50 Kennedy, D. 50 Kepplinger, H. M. 152, 157

Kersting, K. 159 Khan, K. M. 50 Khera, V. 100 Kiernan, V. 2-3, 26, 34, 50, 80, 97, 101, 118-119, 124, 131, 133, 137 Kieslich, P. J. 101 Kiesslich, T. 122, 137 King, D. W. 103 King, S. 102 Kinnebrock, S. 158 Klamer, A. 98, 99, 101 Kleinert, C. 100 Koch, D. 156 Koguchi, T. 82, 101 Kohler, S. 147-148, 151, 153, 157 Kohring, M. IX, XVI, 140, 142, 157, 158 Konkiel, S. 124, 131, 137 Konsman, J. P. 157 Koppers, L. 153, 158-159 Kosicki, G. M. 56-57, 77 Kousha, K. 82, 101 Kreps, S. E. 54, 77 Krieken, R. van 98-99, 103 Kriner, D. L. 54, 77 Krippendorff, K. 90, 101 Krueger, R. A. 85, 101 Kubli, M. 157 Kulkarni, A. V. 87, 101, 104 Lalu, M. M. 50 Lammey, R. 75 Langfeldt, L. 136 Larivière, V. 77, 137 Latour, B. 1, 26, 57, 77 Laukötter, E. 49 Lauro, M. 75 Lawson, B. 51 Le Masurier, M. 53, 57, 77 Leach, J. 158 Leeuwen, T. van 76 Lehmkuhl, M. XII, 139-141, 144, 147-148, 151-159 Leidecker-Sandmann, M. XII, 139, 141, 153-154, 157-158 Lemke, S. VII, XII, XIV, 3, 5, 26, 80-83, 96, 101-102, 117, 119, 123-124, 131, 133- 134, 137

Lentsch, J. M. 146, 159 Lersch, E. 158 Leßmöllmann, A. VII, XV-XVI, 49 Lester, M. 33, 50 Levchenko, M. 75 Levine, K. J. 103 Lewenstein, B. V. 55, 77, 158 Leydesdorff, L. VIII, XVI Li, D. 50 Li, W. 43, 50 Li, X. 137 Liang, X. 120, 137 Liewehr, D. J. 101 Lightfoot, A. 137 Lin, M. 138 Lindsay, N. 138 Lissauer, G. 158 Löffelholz, M. 47, 157 Loosen, W. 30, 50 Lorenz, L. 50 Lorke, J. 50 Louviere, J. J. 81-82, 97, 102 Löwe, A. 76 Luhmann, N. 142 Luschmann, E. 159 Lüthje, C. XVI, 51 Lyne, J. R. 45, 50 Mabizela, M. 50 MacCallum, C. J. 138 MacKerron, G. 52 Madjarevic, N. 137 Maeda, S. 82, 101 Maggio, L. A. XV, 136 Manca, A. 50 Mandelbrot, B. 147 Marcinkowski, F. 49, 142-143, 155, 158 Marres, N 25 Marshall, D. 87, 102 Martín-Martín, A. 23, 26 Marzinkowski, H. 76 Massarani, L. 158 Matheson, D. 29, 50 Matthias, L. 57, 77 Mayr, P. 136 Mays, R. 103 Mazarakis, A. VII, XI-XII, XIV, 79, 102, 117

Mazzolini, R. G. 146, 156 McCartan, K. 85, 102 McVeigh, M. E. 100 Mede, N. G. VIII, XVI, 77 Mehrazar, M. 102 Mendonça, A. 75 Merten, W. 50 Mertens, U. 101 Metag, J. 157 Metcalfe, J. 50 Meyen, M. 145, 158 Meyer, M. VIII, XVI Miettunen, J. 87, 102, 105 Milde, J. XVI, 51 Mitomo, H. 101 Modigliani, A. 45, 49 Moher, D. 50 Moles, A. A. 31, 50 Molina Ramirez, A. 159 Molotch, H. 33, 50 Momeni, F. 136 Mongeon, P. 77 Monson, J. Q. 100 Montori, V. M. 100 Moore, C. A. 55, 77 Moorhead, L. XV, 49, 76, 136 Morgan, D. 138 Mühlemann, B. 77 Müller, B. 50 Müller, S. 157 Münder, H. 100 Na, J. C. 96, 101 Nanni, F. XV, 49, 76 Nealey, P. 137 Nehring, L. 136 Nelkin, D. 31, 51, 55, 77 Neuberger, C. VII, XVI, 30, 51 Neuberger, M. M. 103 Nevill, T. 124, 131, 138 Newman, M. 148, 158 Neylon, C. 137 Nguyen, A. 41, 51 Nguyen, T. N. 136 Nielsen, K. H. 139-140, 158 Nieminen, P. 87, 102, 105 Nieri, M. 87, 102, 104-105

Nisbet, M. C. IX, XV Nishimura, K. 92, 99 Nölleke, D. 51 Noorden, R. van 82, 103 Nosek, B. A. 77 Nowotny, H. 31, 51, 144, 158 O'Connor, D. J. 25, 100 O'Neill, D. 36, 50 Odom, A. R. 99 Oehmer, F. 156 Oliveira Henriques, S. 138 Orduña-Malea, E. X, 1, 9, 26 Ortega, J. L. 5, 26 Oulif, J. M. 50 Overholser, G. 50 Pagliaro, U. 102 Pálfy, M. XV, 49, 76 Pan, Z. 56-57, 77 Pansegrau, P. 140, 159 Parilli, C. 75 Parkin, M. 75 Parviainen, J. 54, 77 Paterson, C. 48 Paterson, Q. S. 138 Patil, S. 102 Patsopoulos, N. A. 87, 102, 104 Patterson, T. E. 34, 44, 51 Pellegrini, G. 75 Penfold, N. 75 Pereira, P. 75 Peters, H. P. 30, 51, 140-141, 158 Peters, I. VII, XI-XII, XIX, 26, 79, 102, 117, 136-137 Petkova, K. 156 Pfeiffer, N. 75 Phillips, D. P. VIII, XVI, 2-3, 26, 80, 83, 97, 102, 119, 124, 131-134, 137 Pinfield, S. 76, 138 Pini-Prato, G. 102 Piwowar, H. A. 137 Plume, A. M. 100 Pöhler, L. 102 Polka, J. K. XV, 49, 75-76 Pollitt, A. 83, 85, 102 Potoglou, D. 102

Pranz, S. 52 Priem, J. 120, 124, 131, 137 Pringle, J. 100 Promies, N. XII, 139, 147, 148, 151-155, 157-158 Pröschel, L. 29, 32, 47-48, 81, 96, 99-100 Puebla, I. 75 Quintanilla, M. Á. 25 Raeymaeckers, K. 57, 77 Rafols, I. 101 Rahnenführer, J. 159 Rauchfleisch, A. 151, 159 Raupp, J. IX, XVI, 30, 51 Reed Johnson, F. 102 Regorz, A. 121, 124, 138 Reinemann, C. XVI, 51 Renn, O. IX, XVI Repper, F. C. 43, 49 Rhomberg, M. XVI, 30, 50-51 Richens, E. 76 Riedlinger, M. 49, 76, 158 Rieger, O. Y. 75 Rijcke, S. de 25, 101 Riles, J. M. 58, 77 Ritter, W. E. 31, 51 Rittman, M. 75 Robinson-Garcia, N. 78 Robson, C. 85, 102 Rocha da Silva, P. 34, 52 Rödder, S. X, XIX, XVI, 25-26, 29, 31, 47, 49, 51, 136, 157, 159 Rosen, J. 145, 159 Rössler, P. 148, 159 Rotgeri, S. 26, 99-100, 135, 137 Rousseau, R. 26 Rubin, A. 75 Ruge, H. M. 98, 101 Ruß-Mohl, S. 37, 51 Ryfe, D. 33, 51 Rzayeva, N. 138 Safipour Afshar, A. 103 Sakmann, J. 26, 137 Saletta, D. 102 Sampson, H. 100

Sanders, J. L. 138 Sangalang, A. 77 Sansone, S. A. 138 Satorra, A. 124, 138 Schaer, P. 100 Schäfer, M. S. VIII-IX, XVI-XVII, 26, 151, 155, 157, 159 Schäfer, S. 30, 51 Schemitsch, E. H. 100 Scheufele, D. A. 137, 140, 159 Schiele, B. 157, 158 Schimank, U. 143, 159 Schindler, J. XVI, 51 Schlesinger, P. 45, 51 Schlögl, C. 124, 138 Schmid, P. 52 Schmitz, C. 85, 102 Schneider, M. 50 Schniedermann, A. X, 53 Schudson, M. 33, 51 Schuir, J. 82, 102 Schultz, I. 37, 51 Schulz, P. XV, 49 Schwartz, L. M. 2, 27 Scott, P. 51 Seeley, M. K. 99 Serger, H. 48, 52 Serong, J. 155, 159 Sever, R. 75 Shams, I. 101 Shevchenko, Y. 101 Shi, S. 157 Shi, Y. 50 Shinn, T. VIII, XV Sidler, C. 101 Simons, A. XI, 53 Sjøvaag, H. 57, 77 Slocombe, D. 50 Smith, A. 76 Soderberg, C. K. 77 Solomon, D. J. 39, 52 Sovacool, B. K. 39, 52 Spohn, M. 75 Stadel, F. 52 Steeg, J. 138 Stehr, N. VII, XVI Stein, A. 77

Steinberg, S. M. 101 Steiner, A. 142-143, 155, 158 Stirling, A. 52 Stocklmayer, S. M. 25, 100 Stockton, N. 2-4, 7, 26 Stonbely, S. 33, 52 Strauss, A. 62 Streiner, D. L. 121, 138 Stroobant, J. 57, 77 Stryker, J. E. 123, 131, 138 Su, L. I. F. 137 Suhr, M. 29-30, 32, 46, 48, 52 Swaminathan, S. 75, 138 Swiontkowski, M. 100 Szigeti, A. 158 Tahamtan, I. VIII, XVI, 81, 84, 86, 103-115, 118, 121-123, 131, 138 Tandoc, E. C. 30, 33, 52 Taraborelli, D. 137 Tastad, P. L. XVI, 26, 102, 137 Taubert, N. C. IX, XVI, 39, 52 Tenopir, C. 79, 81-84, 103 Teuteberg, F. 102 Tewksbury, D. 77 Thelwall, M. 26, 82, 101, 124, 131, 137-138 Thoma, B. 124, 131, 138 Thomas, J. 50 Thurman, N. 43, 52 Thurman, P. W. 101 Tian, D. 54, 78 Till, J. E. 54-55, 78 Togeby, L. 156 Tornetta, P. 100 Torres-Salinas, D. 54, 78 Traag, V. A. 76 Traweger, A. 137 Trench, B. XV, 30-31, 48-49, 52 Trenn, T. J. 49 Tuchman, G. 33, 52, 56-57, 78 Tunger, D. 100 Udris, L. 156 Usher, N. 30, 48 Väliverronen, E. XVI, 30, 52 Veith, T. 77 Velen, V. A. 50

Visser, M. 23, 26 Vogler, D. IX, XVII, 2, 4, 26, 155-156, 159 Voigt, M. 76 Vrieze, J. de 25, 125, 138 Wagner, H. U. 158 Wagner, R. 52 Wahl-Jorgensen, K. 30, 50, 52 Walker, R. 34, 52 Walter, G. 52 Waltman, L. 26, 76, 101, 124, 138 Wang, J. J. 50 Wang, P. 54, 78 Warthun, N. 50 Watson, C. XIV, XVII Wears, R. L. 100 Weaver, D. H. 47 Weber, E. J. 100 Weiber, R. 100 Weigel, D. 76 Weingart, P. IX, XIV-XVII, 2, 25-26, 47, 49, 55, 78, 134, 136, 138-143, 146, 155, 157, 159 Wells, R. XV Wen, R. 75 Westlund, 0. 29, 33, 36, 44-45, 49, 52 White, C. 136 White, H. D. 98, 103

Whitley, R. D. XV Wilhelm, J. 148, 159 Williams, A. 31-32, 52 Willis, D. L. 87, 103-104 Wilson, C. 99 Woloshin, S. 2, 27 Wooding, S. 102 Woolgar, S. 1, 26 Wormer, H. XIII-XIV, 34, 36, 38, 47, 134, 136, 156, 159 Wouters, P. 5, 27, 101, 136 Xenos, M. 137 Xiao, L. 50 Xiao, T. 50 Yang, L. 50 Yao, B. 50 Yeo, S. K. 137 Zahedi, Z. 27 Zhao, X. 51 Zimmermann, G. 137 Zinke-Wehlmann, C. VII, XIV Zotta, F. 50, 159 Zuchowski, M. 77 Zuydam, L. van XV

#### **Subject Index**

Academic Mozdeh 8 access 33, 62-65, 70, 72-74 accessibility 55, 59, 66, 69, 82 accountability 57 actor 6, 15, 21-23 – fiction 155 – theory 143 actuality 36 agenda setting 41, 43, 45, 81 AlphaGalileo 3, 23, 34 altmetric attention score 80 Altmetric.com 5, 58, 127, 133-134, 147- 149 altmetrics 24, 82, 96, 120, 123, 125, 127, 131, 135 American Association for the Advancement of Science (AAAS) 4, 125 Annotated Publication List 41 applicability 37 article 1, 5, 22, 57, 65, 79-80, 82-83, 98, 117-118, 120 – attribute 81, 84, 86-87, 94, 96, 98, 135 – type 12, 20 arXiv 55 Associated Press (AP) 56, 77 astonishment 38 attention 80, 98-99, 117-118, 120, 124, 143, 145, *see also* public awareness – concentration 146, 148 – distribution 33, 121 – economy 98-99 – scientific 154 audience 2, 155 authority 57 awareness 79, 83, 99 axial coding 33 beat system 34, 46

BeautifulSoup 60 bibliometrics 125, 133, *see also* scientometrics

BILD 60, 72 blog 6 broadcasting format, *see* Science Media Center (SMC) choice set 84, 87, 90 circulation 59 citation 6, 80-81 – advantage 80, 132 – analysis 81, 117 – context analysis 75 – count 80, 82-84, 96-97, 99, 118-121, 123, 125, 128, 132, 134-135 – norm 135 cognitive effort 98 cognitive window 57 collective sense-making 40, 45 comparative fit index 129 Competence Centre for Bibliometrics (CCB) 127 conditional logistic regression model 92, 96-97 congruence-frequency distribution, *see* power law conjoint analysis 81-84, 87, 92, 96 construction of reality 33 constructivist grounded theory 33 Corona, *see* COVID-19 COVID-19 33, 35, 41, 53-54, 59, 61, 65, 67, 69 credibility 38

Delphi study 84-87, 96 democracy 31 design 94 Deutsche Presseagentur (dpa), *see* German Press Agency digital media 42 digitalization 120 discrete choice experiment 81 dissemination of research findings 33, 55 DOI 8, 12, 19-20, 22, 68, 125-126, 133 download count 35 earmark hypothesis 2, 80-81, 97, 99, 119, 132 editorial hierarchy 46 editorial meeting 35, 43 embargo 3-4, 34, 40, 42, 46, 118, 120, 133 – e-mail 3, 34, 120-123, 125-126, 128, 131- 133, 135 – time advantage 35 epistemic authority 45 epistemisation of politics 146 epistemology of journalism 29 ethnography – approach 32 – newsroom study 29 EurekAlert! 3-4, 7, 11, 21, 34, 125-126, 132-134, 149 – keyword 22 events 45 expert statement 29, 31, 41-42 expertise game 54 external science communication 118, 120- 121, 125-126, 131-132, 134 Facebook 120 Factsheet 40 factuality 56 fatigue 90, 94 Fedorov's exchange algorithm 89 feedback 33, 145, 150-153, 155 fieldwork 33 filter 33 Flow Metrics – Citation Flow, *see* Majestic – Trust Flow, *see* Majestic focus 72, 74 focus group 85-87, 97 focused coding 33 fractals 147 framing 54, 57, 62, 64, 69-70, 72-75

gap between public and scientific sphere 140-141 gatekeeper 99 genre 69, 71-74 German Press Agency 36 German Press Council 71 Germany 71, 153 gut feeling 37 heterogeneous coupling 6, 19, 22 heuristic 94-97 Ida (fossil) 144 immediacy 57 impact 5-6, 16, 22-23, 81, 133, *see also* citation count impact factor, *see* journal impact factor importance 94, 96 Information Exchange Group 55 Informationsdienst Wissenschaft 34 Ingelfinger rule 3 innovation 83 institutionalized procedure 30 interaction 19, 22, 24, 81 intermediary 30 internal science communication 1, 19, 55, 82, 97, 99, 118, 121, 141, 156 interpretation 33 journal 19-21, 45, 55, 127, 151-155 – editor 3 – multidisciplinary 127, 134 – reputation 121-122 – quality 38 journal citation median 128 journal impact factor 38, 82, 120-123, 132, 134-135, *see also* journal reputation journalism 29-31, 46, 57, 139-140, 142- 145, 147, 150, 152 – double role 145 – media 120 – social significance 141, 143, 145 – research 32

journalist 2-4, 31, 34, 41-42, 54, 69-70, 74, 80, 140 – expertise 38 – German 65 journalistic selection process, *see* selection process journalistic sphere 29, 44, 46, 120, 147- 149, 151, 155

Klaus Tschira Foundation (KTS) 32 knowledge broker 32 knowledge society 143 knowledge-based journalism 44

lavaan (R-package) 121, 130 legitimacy 57 legitimazation 54 LimeSurvey 85

mainstream media, *see* mass media mainstream media mention, see Altmetric.com Majestic 8-10 mass media 5, 121, 139, 141, 143-144, 150-151, 156 mass media mention, see Altmetric.com Matplotlib 60 Matthew Effect 98 MAXQDA 33 media attention, *see* media coverage media attention curves 37 media coverage 2, 35, 37, 54, 58, 84, 96- 99, 118-119, 122-123, 125, 128, 131- 132, 134-135, 139, 145-146, 148-150, 153-156 media effects research 145 media exposure, *see* media coverage media logics 146 media observation 146 media outlet – Canadian news 58 – Danish newspaper 150 – German news 34, 53, 55, 58, 60, 70, 72- 73 – German newspaper 147-148, 150, 154 – news 2, 79, 82-83, 96, 98

– newspaper 59, 79-80, 119 – online news 59 – online newspaper 147-148 – Swiss newspaper 151 – US news 58 media presence, *see* media coverage media system 143 medialisation 45-46, 141-146, 154-156 – of politics 150 – of science 134, 141 median-based impact factor 122 mediation 139 metadata 59 MeWiKo research project 87 mock-up news report 87-89, 91, 95-97 model fit 130 msm-score, *see* Altmetric.com multiple regression 121 naming 64-65, 72 Nature 127-128 neoliberalization of scientific research 30 New England Journal of Medicine (NEJM) 119, 122 New Public Management 143, 155 New York Times (NYT) 80, 119 news clipping 43 news content 124 news coverage, *see* media coverage news discourse 57 news factor 33-34, 36, 44, 98, 152, 155 news production 30, 33, 40, 44, 46 news story 56-57, 59-61, 71-72 news value, *see* news factor Newsdesk 34 newsmaking routine 29 newsroom 30, 32 newsworthiness 70. 72, 80, 81, *see also* news factor Nexis 149 novelty 38, 118 objectivity 44 object 6, 11, 21-22, 24 observation 33

obsolescence 17

online impact, *see* altmetrics open access 38 open coding 33 open science 39, 120 OpenRefine 8 ordinal logistic regression model 97 Pandas 60 part-worth model 82 path analysis *see* path model path model 117, 121, 123, 131-132, 134 paywall 59 peer review 1, 35, 38, 54-55, 69, 74, 155 – lack of 56, 69 personalized content 43 persuasion 56 PIOs 4, 16, 21 PLOS ONE 127-128 PNAS, *see* Proceedings of the National Academy of Sciences (PNAS) policymaking 54, 70, 75 political communication 142, 150 political decision-making 37 politicisation – of preprints 75 – of science 146 Postman 8 power law 148 PR officer, *see* science communicator predatory publishing 39 preprint 34, 41, 53-54, 58, 60, 62, 64-66, 69, 70, 72, 74 – server 35, 54-55, 61, 71-72 – traceability 68 press briefing 29 press calendar 36 press coverage, *see* media coverage press database 35 press monitoring 35, 43 press office, *see* science communicator press release 3-5, 7, 11, 14, 21-22, 29, 34, 96, 119, 121-123, 125-126, 128, 131- 133, 135, 149 – theme 12, 21 – type 11, 89 prestige, *see* reputation

priority claim 55 Proceedings of the National Academy of Sciences (PNAS) 128 production of reliable knowledge 142 promotion 23, 117, *see also* science PR public awareness 2, 31, 45, 140-141, 143, 145, 147, 149-152, 154 Public Information Officer (PIO), *see* science communicator public issue 36, 45-46 public sphere 140, 142 public understanding of science, *see* public awareness publicity 143 publicity hypothesis 2, 80-81, 84, 97, 99, 119, 132 Publikation, *see* article publisher 3, 34, 118, 126 Python 60 quality 81, 83, 94, – control 55 – lack of 39 – methodological 94, 96 radio 148 ranking task 96 reach 80-81 recency, *see* timeliness referencing 72 – to preprint 55, 68, 71 – to scientific publication 57, 67, 133, 146 relevance 36, 80-81 – judgment 45, 80 – theory 98 reporting 32, 40, 43, 56, 57, 61, 73, 75, 83, 97-98, 125, 135, 145-146, 155 reputation 2, 38, 47, 82, 98-99, 121, 132, 135, 141 request 60 research article, *see* article research design 96 research evaluation 121 research publication, *see* article retrieval test collection 80 retweet 43

rigor 83, 118 routine 30, 32-33, 44-46 rule 30 sample size 83, 94 scholarly communication, *see* internal science communication SciDev.Net 31 science, *see* scientific sphere Science (journal) 127 science agency 4 science communication 2-3, 5-6, 19, 21- 24, 30-32, 46, 55, 69-70, 79, 81-83, 97, 99, 120-121, 135, 140-141, *see also* external science communication, *see also* internal science communication – strategic 30, 46 science communicator 2 – PR-department of scientific organisation 155 – press office 6, 23, 119 – public information officer (PIO) 4, 15, 23, 133 science influencer 99 science journalism 30, 32, 34, 44, 46, 53, 71-72, 74, 81, 99, 122, 141, 143, 148, 154-155 science journalist 6, 30-31, 43, 56, 118, 133, 141 Science Media Center (SMC) 29, 31, 99 – agenda 36 – broadcasting format 30, 40 – Germany 29, 31-34, 36, 38, 43-45, 81, 87, 125, 131-134 science news 2, 23, 29, 31, 32, 45, 46 – curated 38, 44 science newsroom 35 science publication, *see* article science publishing 70, 72 Science Response 40 Science Service 31 scienitifc sphere 5, 21-22, 72, 120, 132, 140, 142 scientific article, *see* article

scientific expert 38, 41-42, 81, 96, 125, 141, 146, 150-152, 154-155 scientific publication, *see* article scientific sphere 29, 44, 46, 81, 139, 140 scientometrics 4-5, 21, 23-24, 127 Scopus 19, 75, 147, 149, 151 selection process 146-153 – criteria 33, 133 – journal 155 – journalist 80-81, 98, 119, 123 – press office 122, 134 – publisher 131 – researcher 80-83, 86, 88, 94, 96, 98, 134 – science journalism 36, 96 – Science Media Center 36, 39-40, 45 SMC, *see* Science Media Center social media 3, 6, 30, 81-82, 120-121, 123- 124, 144 social media presence, *see* altmetrics socialisation of science 143 SocSciBot 7 speed 57 speed-accuracy tradeoff 57 SPIEGEL 59 Spiegel Online (SPON) 59 statistical literacy 41 stimulus material 84, 90, 96 story, *see* news story structural equation modeling 135 success breeds success 98 supra-individual phenomenon 143-144, 150 t.co shortener 9 The Conversation 23, 31 third mission 120 timeliness 55, 57, 62, 64-66, 70, 72-74 topic 36-37, 82, 133, 152-154 – careers 37 – selection 39 – SMC 44 transparency 57 trend 2, 143 trust in science 143 trust broker 32

trustworthiness 39 truth 53, 56-57, 143 tweet 6, 16, 24, 128 Twitter 7-8, 24, 43, 120-121, 132

uncertainty 56, 62, 64, 66, 69-70, 72-74 unexpectedness 36, 98 urgency 81 utility 82, 92, 94-97

value broker 32 Veröffentlichung 65, *see also* article Virtual Press Briefing 41 visibility 30, 119 Vorab-Veröffentlichung 73, *see also* preprint

Vorveröffentlichung 65, *see also* preprint watchdog 2 web 7-10, 24, 55 Web of Science 75, 125-127, 133-134 – subject category 127 webometrics 9, 24 website 16, 24 working paper 66, *see also* preprint ZBW - Leibniz Information Centre for

Economics 91 ZEIT 59 Zeit online (ZON) 71